RCfORT RESUMES 

ED 017 108 ss em oos sss 

1NTESRAT1N6 PR06RAMED INSTRUCTION WITH CONVENTIONAL CLASSROOM 
TEACHINS. 

«Y- SOLDBECKi ROBERT A. AND OTHERS 

AMERICAN INST. FOR RESEARCH IN BEHAVIORAL SCIENCES 

REFORT NUMBER AIR-C49-1E-6E-FR PUB DATE DEC SB 

REFORT NUMBER 7-S63 

GRANT OEC-73S1S4-01 

EDRS FRICE MF-S0.50 HC-S4.80 IISP. 

DESCRIPTORS- ♦PROGRAMED INSTRUCTION, ^CONVENTIONAL 
INSTRUCTION, #RESPON$E MODE, QUESTIONNAIRES, 4ACHIEVEMENT 
TESTS, ♦PROGRAM DEVELOPMENT, LESSON PLANS, PILOT PROJECTS, 
APTITUDE, AMERICAN GOVERNMENT (COURSE), HIGH SCHOOL STUDENTS, 
SAN MATEO UNION HIGH SCHOOL DISTRICT 

THREE PHASES OF THIS PROJECT SOUGHT TO INTRODUCE 
PROGRAMED INSTRUCTION INTO A ONE MONTH UNIT OF A REQUIRED 
HIGH SCHOOL GOVERNMENT COURSE WITHOUT RADICALLY DISTURBING 
CONVENTIONAL TEACHING AND ADMINISTRATIVE PROCEDURES. PHASE I 
DETERMINED, EXPERIMENTALLY, SUITABLE PROGRAMING TECHNIQUES. 
PHASE II INVOLVED ONE YEAR OF PILOT EXPERIMENTS IN THE 
CLASSROOMS, AND INCLUDED DEVELOPMENT OF CRITERION TEST ITEMS 
AND WRITING OF DETAILED LESSON PLANS. PHASE III, THE ACTUAL 
FIELD TEST, USED TWO TEACHERS FOR PROGRAMING AND 
NON-PROGRAMING CONDITIONS IN CLASSES OF TWO APTITUDE LEVELS. 
STATISTICAL ANALYSIS OF QUESTIONNAIRE AND TEST RESULTS SHOWED 
THAT PROGRAMING COMBINED WITH CONVENTIONAL INSTRUCTION 
SIGNIFICANTLY INCREASED THE AMOUNT OF LEARNING AT BOTH 
APTITUDE LEVELS AND FOR BOTH TEACHERS. STUDENT ATTITUDES 
TOWARD PROGRAMED INSTRUCTION WERE POSITIVE. APPENDICES TO 
THIS DOCUMENT INCLUDE EXPERIMENTAL MATERIALS. (LH) 



4 



EMcoS 



C^3 




AZR-0l»9-lfi/<fie<«R 



■P/9 



Tmasmm mmmp mmxmxm 
Mvm ooroapiowAt cusaRooH umsam 



Robert A. QoISbeck 
Jims W» Shearer 
Peggie Xi* Chnpeau 
Kary B, VlXUe 



Deeeiber X9S2 



U. S. Office of Education 
Oepartnent of Bealth, Edncation, and Welfare 




American Institute for Research 

PENINSULA OFFICE 



8 Watt 41st Avsnus, San Matso 




California 



Flrotido 1-9297 



IWTEGRATIHG PROGR/MED INSTRUCTION 
WITH CONVENTIONAL CLASSROCM TEACHING 

Robert A. Goldbeck 
James W. Shearer 
Peggie L* Campeau 
Mary B. Willis 

AIR-C49-12/62-FR 

Final Report 

Grant No.: Title VII 736134.01 
(Mew Grant Mo.: Title VII 7-48-0000-134.1) 
Leslie J. Briggs, Principal Investigator 



December 1962 



S. Office of Education 

Department of Health, Education, and Welfare 



Main Offica 410 Ambareon Avanva, Pittsburgh 32, Pannsylvania 



ACKNOWLEDGMENTS 



The studies described In this report were made possible largely 
through the excellent cooperation of the San Mateo Union High School 
District (Hr. Thomas F. Reynolds, Superintendent). Capuchlno High 
School and San Mateo High School In this district participated In some 
of the preliminary studies, as did schools In Culver City, California, 
concerning comparison of programing methods for social science material. 
The classroom pilot studies of method combinations and the final field 
test were conducted In Capuchlno High School ^r. Paul'Claudon^ 
Principal) . Serving as coordinator of the research for the school 
district was Dr. M. H. Wlnward. Mr. Leo J. Ryan and Mr. Adolph Harrison 
taught the classes In which the major portion of the experimental work 
was accomplished. These two teachers also made many excellent con- 
tributions to the planning and development phases of the work. 

Some of the preliminary phases of work under this project were 
conducted by Mr. D. G. Nichols, with the assistance of Mrs. Dorothy S. 
New. Mr. Donald F. Terry assisted with some of the data analysis, and 
Mrs. Ethel Kellett assisted greatly In preparing the materials for 
classroom use. 



FOREWORD 



The General Role of Programed Instruction In Education 



The total gamut of current efforts to apply teaching machines and 
programed instruction may be classified in accordance with the intent to: 

a. Make programed learning the sole means of instruction. 

b. Integrate programed learnii^g with other instructional methods. 

c. Automate both the instructional and administrative aspects 
of school operations. 

d. Use machines and programs as a device to promote sale of 
other materials. 

An illustration of intent (a), above, would be use of programs as 
assignment units in correspondence courses in which there is no face-to- 
face interaction between student and teacher. For adult learning, if 
iMe of programs results in higher test scores than other methods of 
home study, this would appear to be an effective method of instructor- 
less training. 

An illustration of intent (c), above, would be the ultimate 
development of an automated school system such as that envisioned by 
Ramo (In Lumsdaine and*.Glaset; ;t960), in which most of the instructional 
as well as the assignment and record-keeping functions of the school 
%K)uld be automated. Needless to say, if such an outcome does develop, 
it is in the far distant future, and whether desirable or not, the 
achievement of such a system would have to await first the further 
validation of computer-controlled instructional systems such as now 
are in an experimental stage (Coulson, 1962). 

Concerning intent (d), above, we will omit comment here, and will 
now announce that the project described in this report was based on 
intent (b) which, in the writer's opinion, is the most fruitful area 
for research and development directed toward immediate educational 
applications. 



The Purpose of the Project 

The project reported here was based on the assumption that for the 
foreseeable future the teacher would and should have a central role 
in education of children in the public schools. It is therefore 
desirable to examine the means by which programed learning can be 
integrated with conventional teaching procedures in such a manner that 
the new and the conventional procedures are mutually facilitating in their 
effects upon achievement of educational objectives. 



In our initial atatementa of project objectivea we aometinea 
referred to our goal aa that of developing an ''optimum'* ayatem for 
introduction of teaching machine techniquea into the claaaroom. Our 
deletion of the word "optimum ' from the title of thia report reflecta 
more than our increaaing humility. It reflecta alao the fact that we 
wiahed to conduct thia reaearch under the adminiatrative conditiona 
now typical of achool ayatem operations ao that the reaulta would be 
applicable inmediately to present school systems. 

Aa a consequence of the above decision, the research had to 
proceed without waiting to benefit from the kind of administrative 
and procedural changes in school operation which would ultimately 
take place and would further enhance the effectiveness of programed 
instruction. We refer to such changes as complete flexible scheduling 
Which would permit (a) students to begin and complete courses as 
their rates of progression warranted, (b) ready formation of homogeneous 
discussion groups based on progress in programed learning, and (c) set« 
ting levels of attainment or mastery to be reached by all students, 
with time-to-mastery rather than amount* learned the measure of individual 
differences. Neither did we choose to wait for large buf^gets, better* 
trained teachers, or more equipment. Although all the above kinds of 
changes, when they tike place, presumably will have the effect of 
requiring modifications from the procedures employed in this project, 
the results of this project, on the other hand, may suggest ways to 
Implement programed learning under those improved conditions. 

In summary, the goal of the present project has been to work within 
present administrative constraints in developing a workable procedure 
for integrating programed instruction with conventional teaching pro* 
cedures. This project may thus be viewed as one contribution to the 
larger problem of exploring the role of the teacher and ways in which 
teacher roles may change as the advantages of programed instruction 
become a reality. 

A course in high school social science was chosen for the purpose 
of the project. A one*month unit of the course dealing with the 
U. S. Federal Government formed the basis for the major experimentation. 



The Phases of Project Work / 

Wbrk on this project eas divided into three main phases. Phase I 
consisted of experimental comparison of alternate programing techniques 
and formats for the presentation of social science materials. Experlmen* 
tal programs on the structure and function of the United Nations were 
employed in this phase in order to (a) protect security of the U. S. 
Government programs until major experimentation was to be undertaken; 

(b) utilise material similar in nature to the U. S. Government materials; 
and (c) be able to conduct two* or three*day experiments with a small 
but meaningful unit of appropriate subject-matter. 

Phase II of the research consisted of brief classroom experiments 
designed to obtain preliminary estimates of several alternate procedures 
for combining study by programs with frequently*used conventional 
procedures. These pilot studies also permitted further analyses of the 
relative merits of several styles of programing. 
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Phase 111 consisted of a field test of the composite Instructional 
procedure which was developed on the basis of total results from earlier 
phases of work. 



»Some Underlying Assumptions 

Throughout the course of work on the project, several findings from 
earlier studies formed a sort of ''apperceptive mass" which probably acted 
as a set of directing tendencies on the part of project personnel. 

First, we assumed that combined methods of instruction were preferable 
to a single method. This preference, in turn, was based upon an expecta- 
tion that there are probably differences among students and among subject- 
matters with reference to optimum mode of presentation. At least one 
set of results (Campbell, 1962) had suggested that a variety of presenta- 
tions may stimulate students to think as they study, and that the nature 
of the material makes a difference in the effectiveness of various pro- 
graming styles. Also, since a certain amount of repetition is assumed 
desirable, it was thought best that some of the repetition be between 
methods and not all within a method. Variety in the day's procedure 
was another element assumed desirable. Finally, i’t was suspected 
that teachers (probably like programers) have "pet" topics which they 
teach better than others. Covering similar material ly two methods 
would then, in probability terms, be called for. 

Second, we believed that a program difficulty level should be set 
so as to require effort (in terms of thinking, re-reading and response 
latency) of the student to (roduce the correct response (Goldbeck and 
Campbell, 1962; Goldbeck and Briggs, 1962). Therefore, variations from 
standard program procedures were tried out both for write-in programs 
and for multiple -choice programs. Both difficulty and format charac- 
teristics of programs were varied in the preliminary studies. 

Third, we suspected that programs developed in cooperation with 
the teachers of the course would have some advantages over programs 
brought in from the 'outside." Some confirmation of this appeared in 
the results in that topics covered adequately by both the teacher and 
the program resulted in high performance on relevant test items. The 
same results suggested that lack of matching of topics by the two 
methods of presentation produces lower test performance. Thus, if 
the teacher helps select the program topics, he is more than likely 
to use those in his own procedures. 

Fourth, we suspected that one of the advantages of programs is 
that they keep attention from wandering as compared to study of text- 
books. This, coupled with lack of ambiguity concerning what the student 
is supposed to do, should make programs successful when used as homework. 
Programs were used as homework in this study with a high percentage of 
conqpleted assignments, and consequently participation in class by ttie 
students was 'improved . 

Fifth, we suspected that prior successful use of programs to 
reinforce main points presented by demonstrations,^ films, field trips, 
etc. (Briggs, 1962), implied that, were films to be used in the experi*^ 
n»ntal course unit, ways of drawing attention to the main points were 
needed . 
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Finally, na believed that a lalrly direct emphasis upon the necessity 
for completing required study can successfully be made When there Is no 
ambiguity as to what the requirements are and when the methods offer 
possibility of success as a result of making the required effort. Through 
the cooperation of the experimental teachers on these points, good results 
were achieved. 



Looking Toward the Future 



As the main body of this report Indicates, the experimental effort 
to integrate programed Instruction with other classroom procedures resulted 
in test scores significantly higher than those earned by control groups as 
Indicated by standard statistical tests of significant differences. 

Judging the practical significance of the gains, of course. Is usually 
more difficult. But it Is Important to notice that the statistical 
differences were-based upon a composite procedure in which three- fourth* 
of the classroom time in the experimental group was spent in conven- 
tional activities, while one- fourth” was devoted to programed learning. 

This fact, of course, is consistent with the analytical finding that the 
lectures and the programs were mutually facilitating. 

With'*n present classroom and school conditions, then, it has been 
demonstrated that programed Instruction can be Integrated with conven- 
tional procedures. It was not within the scope of this study to indicate 
how widely this should become the case in general practice, nor w^t the 
costs would be per unit of Improvement in each case. But the findings. 

It Is hoped, will be useful to educators In making judgements about 
these matters. 



In closing the Foreword to this report, the writer would like to 
state that in his opinion the utilisation of programed instruction by 
imaginative teachers could result In the opportunity for teachers to 
find time to develop new and challenging instructional functions. The 
possibility of placing dependence upon the programs to carry a signifi- 
cant portion of the work load should make some time available to search 
for these Improvements. As an aid in the identification of these 
functions, heretofore necessarily neglected by tb» teacher, more research 
is needed In helping teachers to first try out several alternate teacher 
roles in the utilisation of this new media. It is hoped that this study 
represents a small step in this direction. 



Leslie J. Briggs 
Principal Investigator 
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INTEGRATING PROGRAMED INSTRUCTION WITH CONVENTIONAL CLASSROOM TEACHING 



The purpose of this project has been to investigate and develop 
principles and procedures for the optimum introduction of progr^ing 
into a hi^ school course of instruction without disturbing or changing 
schedules or oLtier administrative features of present school practices. 

Thus 9 the goal was to study the introduction of programed learning into 
classrooms in such ways as are immediately feasible in terms of teacher- 
pupil ratios and other current school conditions. Also, the objective 
was to integrate progran^d instruction with conventional teaching pro- 
cedures. Thus, it was relevant to determine how much programing to 
introduce and how to integrate study by programs with total classroom 
activities. The esqphasis is upon how the teacher functions in relation 
to the use of programs, rather than upon any attempt to teach the course 
by programed instruction as the sole method. 

A U. S. Government course was selected for this research because 
(1) this course is required of all high school students, thereby providing 
a large, unselected population of students, and (2) this course repre- 
sented a relatively untried subject matter area for the use of programing. 

The total research for this project involved three major phases. 

The first phase consisted of research in variations in programing tech- 
nique and format for social science materials. The second phase consisted 
of development and evaluation of alternative means for integrating program- 
ing with normal teaching procedures. The third phase consisted of 
evaluation of the effectiveness of the integrated, program-oriented 
instruction. 



Phase l! Preliminary Studies of Altemdte Programing Techniques 

Early efforts on the project were concerned with exploring the efficacy 
of alternative programing techniques for social science materials. It was 
deemed unwise to embark at once upon studies of combining programs with 
normal procedures, since most prior research regarding program technique 
had been conducted in subject matter areas other than social science. 

Cue to probable differences betvraen physical science and social science 
in regard to hierarchical structure of materials and degree of indepen- 
dence asiong elements of the material, it was thought that studies in 
programing of social science were needed first. 

Presented below are those findings that substantially influenced 
the mainstream development of the system for integrating programed 
instruction into a U. S. Government course. These preliminary studies 
used the structure of the United Nations as subject matter because of 
the apparent similarity between teaching U. S. Government and teaching 
U. N. organisation, and because the latter was a more suitable sise of 
unit for short one- or two-day classroom studies. 



Thft flrat of tha ■tudlea wai intended to evlimte th€i_r»l>tlve 
advanfggi of (1> regular text, (2) reviled text (which Included frequent > 
short (3) programad text (with written reiponiei). and (4) pro* 

grsmsd text requiring only the first letter of words ss responses . In 
addition, multiple-choice questions with chemical feedback (achieved by 
treating the letter preceding the correct alternative with an invisible 
chemical which reacted to fluid in the student's felt-tip pen) were pre- 
sented to all groups of students as an immediate review. All students 
worked on these review items, after completion of one trial through the 
assigned program, until the two-hour total learning period was ended. 

Scores on a short-answer test showed no significant differences 
among the four conditions. It was found, however, that the programed- 
text students performed considerably higher when test answers were the 
same as responses in the program, and considerably lower when the test 
answers were not responses in the program. Also, it was found that the 
multiple-choice review items contributed highly to test performance. 

Other findings of general interest growing out of this first preliminary 
study are presented elsewhere (Briggs, et el , 1962). 

The findings of this first study had two major implications for 
development of the programing teaching package: 

1. If the written-response technique were to be used, it either 
should be (a) restricted to teaching key responses or key 
response-related material, or (b) modified to free the tech- 
nique from its strong response orientation. 

2. The apparent efficacy of the multiple-choice review tech- 
nique should be exp . id to determine its most effective 
use and to confirm itw value as a teaching procedure. 

The examination of the multiple-choice review technique was the 
topic of the second preliminary study , which also used the U. N. as 
subject matter. The general format of the programing required the 
student to read pages of ’’text" and to answer multiple-choice questions 
with chemical feedback. Nine conditions were used to determine how much 
text should be read before answering the multiple-choice review questions 
over this material (size of step), and how many review questions should 
be used after each "step" of text. The results showed a surprising lack 
of positive effect from increasing the number of review items, and from 
increasing the number of text divisions (no statistically significant 
differences). On the contrary, the results did suggest that many smII 
steps in text, each followed by many review items, should be avoided. 

This study implied that if the multiple-choicA review technique were 
to be used, two review items after each page of text would be as effective 
as any other combination, and it would be a convenient format. 

Consideration of these results also led to the last of three 
preliminary studies using the U. N. subject matter. If increasing the 
frequency and number of review items did not increase performance, 
perhaps increasing the contextual content in the " text" portion of the 
progr ^B^n d usinfl [_sentencg__ ^edback for the multiple -choice items might 
prove more effective. This hypothesis was t ested, in the third and final 
prai ^m< nitrv ntudv. bv comparing criterion test performance of a group 
using the chemical -feedback version of the U. N. program with a group 
using an enriched-text/statement-feedback version of the program. 
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"Enrichment" consisted of adding background material, anecdotes, and 
examples to the text portion of the program. Test differences between 
the groups were slight. The implication of this study was that adding 
contextual content to the text material in the programs was not a 
promising way to increase the learning of the basic subject matter . 

(A study just completed under sponsorship of the Office of Naval Research, 
Contract No. Nbnr-3077(00) suggests that different results could have been 
obtained by identifying which statements in the text are "enrichment" 
statements and which are "basic" statements.) Although the statement- 
feedback did not produce any evidence of increased learning, it neverthe- 
less proved to be a convenient format for the multiple-choice technique. 

In suomary, the three preliminary studies using the U. N. material 
had the following major implications fpr development of the programs 
to be used for the program-oriented teaching of government: 

1. Conventional write-in programs should be used only to teach 
material for which the criterion is to answer test questions 
with responses that are the same as the responses used in the 
program, or the programing technique must be modifed to pro- 
duce a better knowledge of the material not called for as 
responses in the program. 

2. The programs consisting of "text" and multiple-choice items 
may conveniently have a ratio of about tw) multiple-choice 
items for each page of text. 

3. Adding "enrichment" material to the text portion of the 
program (without givipg the student a guide for discriminating 
the "enriched" material from the "basic material), or 
increasing the number or frequency of multiple-choice items, 
does not add to performance as measured. 



Phase II: Development and Evaluation of Alternative M eans for Integrating 

Programed Instruction with Normal Procedures 

The following is an account of the second phase of work involving the 
development of the program-oriented instruction for the unit on "The 
Structure and Fimction of the Federal Government" (referred to as 
" Unit in" in this report). l?hen reading this account, it will be 
helpful to keep in mind that the effort in this second phase of work 
involved working with government classes held during two successive 
semesters: 

Fall 1961 : Tryout of test items for Unit III, first conq>arison 

of method combinations, in conjunction with first 
classroom tryout of programs for the last portion 
of Unit III. 

Spring 1962 : Classroom tryout of all programs, in conjunction 

with full-scale methods comparison. 

Selection of the content material for the programs and tests was 
based on the outcome of several efforts. While the procedures used 
were ;sot directly drawn from any one specific source, they were undoubtedly 
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Influenced by reference to iooe of the publications concerned with the 
classification of educational goals (Blootn:, 1956; Gross and Zeleny, 1958; 
Johns, 1953; Patterson, 1960; Carpenter, 1953; Price, 1958; NCSS Conmittees 
on Concepts and Values, 1957; California State Curriculun Jomnission, 1962). 

Initial background material oi the course was obtained by (1) discus- 
sions with the teachers who conducted the classes, (2) examination of the 
course outline, (3) reading the course textbook (Brunts, 1961) and other 
textbooks on U. S. Government (Posey and Htiegli, 1959; Haefner, et 
1960; Brown and Peltier, 1960), (4) examining previously used tests, and 
(5) observing class sessions. 

After the general content to be covered by the course was determined 
by the activities described in the paragraph immediately above, there 
remained to be developed a more concrete plan for the preparation of 
criterion tests, programs, and lectures. Thus, an item-by-item information 
analysis was performed on the material contained in the class textbook, 
the material derived from class sessions, and the information drawn from 
discussions with the teachers. Each item of information was rated on a 
three-point scale of isqportance by one of the teachers. The ratings were 
reviewed to assure their consistency and to confirm that some plausible 
dimension had been used in scaling the items. This dimension can best 
be described as "specificity-generality." That is, items of information 
rated high on the scale were general in nature and had many implications 
for a number of related information items. 

This information analysis of the course content material served as 
a guiding basis in preparing tests, programs, and lectures. With respect 
to test and program development, the function of the information analysis 
was of greater relative importance in selecting test items than in deter- 
mining content coverage in the programs because of the need in programing 
to comnunicate on a broad base of exposition. Nevertheless, the information 
analysis was the only formal link relating the programs to the tests. 

Criterion test development . The criterion chosen for competing the 
rolative effectiveness of various instructional methods combinations was 
students' scores on objective tests based on course content. It is both 
impractical and unnecessary to develop criterion test items which assess 
students' knowledge of and ability to apply everything they have supposedly 
learned about a subject as broad in scope as the structure and function of 
the U. S. Government. There are two main solutions to the problem of 
developing effective criterion measures. The first consists of selecting 
the desired criterion behavior and then focusing instruction and sube?!quent 
evaluation solely on this behavior. The importance of applying this "teach 
the test" approach is extolled by many "experts" but is not typically em- 
ployed by classroom teachers who are involved in teaching any high-density 
subject matter area such as government, history, economics, literature, etc. 
The alternative approach involves attempting to develop criterion measures 
which adequately sanq>le students' knowledge of the material in question. 

The basic assumption in this instance is that if a student demonstrates 
that he has "learned" a given percentage of a sample of course material, 
he also has "learned" this same percentage of the entire mass of course 
material . 

The approach used in the Spring 1962 classroom pilot study was a 
combination of the two approaches mentioned above. Students were "taught 
the test" with regard to the more important aspects of Unit III and were 
tested over the relatively less important aspects by the "sampling" method. 
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Initially, a large pool of objective test items was constructed to 
provide a means of assessing students' knowledge and understanding of 
important factual and conceptual material presented in the five chapters 
of text which constituted one unit (Unit III) of the U. S. Government 
course of study. Each item in the pool of test items was classified as 
being essentially a specific fact, general fact, application, or vocabu- 
lary item* This classification was made so that items selected for a 
test would represent as good a balance as possible among these four 
"kinds" of items, within the constraints of the subject matter. 

Classifying a test item in this manner involved determining what 
that test item das ^intended to measure. Since nearly all items required 
a student to combine his knowledge of specific and general facts, his 
understanding of subject-matter vocabulary, and his skill in applying 
all of this when answering the test item, one could argue that actually 
any test it^ could fall into all four categories. However, by determining 
what the item was intended to emphasize most heavily, that item could be 
assigned its appropriate classification. 



Following is a more detailed account of the rationale used in assigning 
each test item to one of the four classifications: specific fact , general 
fact, application , or vocabulary . 

THEN THE CLASSIFICATION 

IF THE ITEM WAS INTENDED TO TEST THE STUDENT'S: ASSIGNED THE ITEM WAS:.. 

. • • knowledge of subject-matter vocabulary 

definitions VOCABULARY 



• • . ability to apply his accumulated knowledge 
of pertinent subject-matter vocabulary, 
general facts, and specific facts to a 
"new" situation (one not duplicated in the 
text, program, or lecture) 

. . . knowledge of principles, concepts, philos- 
ophies, or ideas which are basic enough to 
underlie or serve as foundations for insti- 
tutions, processes, systems, etc. 

. . . knowledge of factual material which, 
although it may be "basic" to his 
understanding of institutions, processes, 
systems, etc., may not be as broad in 
scope as "general fact" material 



APPLICATION 



GENERAL FACT 



SPECIFIC FACT 



In addition to the above classification, results of the information 
analysis of. course ^rntent material (described previously) served as a 
guide to staff personnel in (1) developing additional test items to insure 
adequate content coverage and (2) selecting items for inclusion in the 
various criterion tests. 



The types of item format to be classified were multiple-choice, true- 
false, and short-answer completion, since the students' previous U. S. 
Government tests included these types of items, and it was intended that 
the experimental tests should be as similar as possible to the regular 
course tests. Prospective items were next reviewed by an experienced 
teacher and by project staff members in order to eliminate or revise those 
which were ambiguous or misleading, and to insure that the items were 
written at a level appropriate for twelfth-grade students. 



5 



Test-item tryouts were first performed in conjunction with prellmi- 
nery progrem tryouts* Individuel studf*nts first conqpleted progrsms end 
then attempted to answer relevant test items. The items then were dis- 
cussed with each student. Next, items from the pool were tried out in 
the Unit III test administered to approximately 225 students in five 
classes during November, 1961. Another group of items was administered 
to the same students in the January 1962 semester examination. Follow- 
ing both review and tryout, items were deleted or modified, and new 
items were added to the original pool to cover important points more 
effectively. 

Estimates of difficulty and internal consistency described by 
Thorndike (1949) were computed for those items which were included in 
the Nbvember 1961 Unit III test and in the January 1962 semester examina- 
tion. The difficulty index served two primary purposes. The first 
purpose was to identify extreme items s i.e., those which are very easy 
or very difficult* The second purpose was to insure that criterion tests 
discriminated reliably bettseen students who learned a high proportion 
of course material and those who learned a low proportion of course 
imf*terial, theieby providing criterion tests which would discriminate 
between instructional methods which produce learning. The consensus is 
that some degree of scatter around an average item difficulty in the 
middle-difficulty range (preferably around .50) will produce sufficiently 
reliable discriminaticn . The average item difficulty for the criterion 
tests used in the Spring 1962 classroom pilot study was .69, with a 
moderate amount of scatter in the difficulty distributions. 

The method used to compute the difficulty index involved (1> deter- 
mining the proportion of students who responded correctly to each item, 
and (2) correcting for chance success using the following formula: 



R■^W-eO 

Pg s the proportion of students actually knowing the answer to the item 

R - the number of students giving the right answer 

W - the number of students giving a wrong answer 

0 - the number of students choosing to omit the item 

n s the number of response alternatives for the item 

Internal consistency data were also used in an editing capacity; 

i.e*, to identify questionable It^s, particularly those having extremely 
low or negative correlation estimates and to insure adequate item discrimi- 
nation. Steps followed to obtain internal consistency estimates were as 
follows: 

1. The high and low 27 per cents of the total sample of students 
were selected on the basis of total score on all items. 

2. The proportions of students in the high and low 27 per cent 
groups who responded correctly to each item were determined. 

3. The above proportions were corrected for chance success (same 
as for difficulty index determination). 
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4» Estlnatet o£ correlation coe££lclenta were obtained by entering 
a table prepared by Flanagan (1962) • 

An analy^la-of -variance technique developed by Hoyt (1941) uaa used 
to compute estimates o£ reliability £or the criterion tests* Reliability 
coe££lclents obtained £or the £ive segment tests were the £ollowlng: 



According to Gull£ord (1936), it is generally agreed that £or evaluating 
individual scores the reliability o£ a test should be at least .90, but 
that £or evaluating group scores the reliability may be as low as .50. 

'Ae coe££icients obtained £or the various segment tests £all well within 
the "adequate” range £or assessing group accomplishment. 

Program development . Two sets of programs were prepared £or the 
Unit III material in the U. S. Government course. The iormats o£ the 
two sets o£ programs were determined in part by t|ie results o£ the three 
preliminary studies using programed material on the United Nations. 

One set o£ programs consisted o£ pages o£ text with two multiple- 
choice questions at the bottom o£ each page. The questions were chosen 
to test comprehension o£ the page of text with the intention to direct 
the student to a rereading o£ the text i£ he could not answer the 
questions. The answers to these questions were g iven in paragraph £orm 
at the too o£ the £ollowing page, above the text on th at page (see 
Appendix K) . This £ormat was derived rather directly £rom the results 
o£ the preliminary studies described earlier. 

A second set of programs over the same material was prepared in a 
’write-in" £ormat. A major modi£ication o£ the conventional write-in 
£ormat used £or the U. N. material was introduced in order to eliminate 
the response-bound nature o£ the U. N. program. The key to this 
modi£ication was eliminating £ormai presentatio n o£ the write-in 
responses as £eedback . The "£eedback" was written into the material 
surrounding the write-in blank instead o£ being presented as an isolated 
answer £or the blank. Students were thereby required to read the 
surrounding context o£ the blanks to determine and to con£irm their 
responses £or the blanks. 

The rationale £or this procedure is consistent with the Senegal 
rationale underlying the £indings reported by Goldbeck and Campbell (1962) 
that write-in response programing was more e££ective when response 
di££iculty was high (as measured by response latency) than when it was 
low. The assumption, in both that study and the present one, is that 
programing can be made more e££ective if the program requires the student 
to stop and think (study) as he proceeds through the program. For^he 
U. S. Government programs, this characteristic w as intended to be 
e££ected bv withholding simple feedback to write-in responses and 
embedding it instead in the reading portion of the program. 



Segment 



Reliability Coefficient 



1 

2 

3 

4 

5 



.82 

.77 

.81 

.75 

.76 
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In addition, tha uaual feature of dividing write-in programed material 
into email etepa or framea, one to a page, waa eliminated. Thia char- 
acteriatic made it poaaible for atudenta to reread and atudy whole pagea 
of program after the blanka had been filled in, the aame way that textbook 
material can be reviewed. An example of thia programing ia ahown in 
Appendix F. 

The initial veraiona of both kinda of programing were tried out with 
a few atudenta on an individual baaia. Theae initial veraiona were re- 
viaed on the baaia of program errora, teata adminiatered to atudenta, and 
inter\*iewa with the tryout atudenta. The two teachera in charge of the 
U. S. Government couraea alao contributed many helpful auggeationa for 
reviaing the programa, baaed in part on their knowledge of thoae portiona 
of the aubject matter which preaent more difficult learning problema for 
the atudenta. 



Fall 1961 Claasroom Pilot Study 

The firat tryout under claaaroom eonditiona of a U. S. Government 
write-in program waa conducted during the Fall 1961 aemeater . The Federal 
Courta write-in program waa reviaed in time to be uaed in two (of the five) 
U. S. Government claaaea without altering the normal curriculum aequence. 
All atudenta were general-level atudenta (college preparatory atudenta 
having been acheduled to take the courae during the Spring 1962 aemeater) . 

Although both U. S. Government claaaea apent two claaa daya on the 
Federal Courta material, the method of combining programed inatruction 
with lecture-diacuaaion over thia material waa different for the two 
claaaea. Claaa A worked on the program on Day 1, had a lecture-diacuaaion 
period on Day 2, and waa teated on Day 3. Claaa B had a lecture-diacuaaion 
period on Day 1, worked on the program on Day 2, and waa teated on Day 3. 
Both theae claaaea cooqpleted an average of about two-thirda of the program 
in claaa, cooq>leting the laat third of the program as homework. 

A third general- level claaa alao apent two claaa daya atudying the 
Federal Courts, but did not use the program. Two other general-level 
classes spent four class days studying this topic without use i^f the 
program. 

On the 18-question test over this material, Class A (with the 
program preceding the lecture-discussion period) answered an average of 
62 per cent of the questions correctly, and Class B (with the program 
following the lecture-discussion period) answered 69 per cent correctly. 
While this difference favored the condition of following the lecture- 
discussion period with a program, the difference was not statistically 
reliable. Analysis of covariance (using the verbal scale of the Differ- 
ential Aptitude Test as the predictor variable) showed no statistically 
reliable differences among the two classes using the programs and the 
two non-program classes which spent twice as much class time (four days) 
on the topic. The latter (non-program) classes scored 66 per cent and 
68 per cent on the test. The non-program class which spent the same 
amount of class time as the program classes scored only 58 per cent 
on the test. This score is reliably lower than the scores for three 
of the other four classes, the exception being Class A. 
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These results with e prelinlnery version o£ the wrlte-*ln program 
were encouraging with reference to the possibility of successfully Inte- 
grating programing Into a classroom teaching-package. In regard to the 
Intent of the programing to cause students to study as they worked 
throu^ the program, about one-third of the students reported that they 
reread part of the material before filling In at least half of the blanks* 
About half of the students reported they only occasionally reread material 
before filling In a blank. 

The data were somewhat ambiguous on the question of Whether programs 
should precede or follow lecture-discussion periods. It Is relevant 
that the program error rate for the class with the program first was 
12 per cent, while the error rate for the class with the lecture-discussion 
first was only 8 per cent. On the other hand, the class with the program 
first responded joiore adequately to the teacher's questions during the 
lecture-discussion period. Students In this class failed to answer only 
18 per cent of classroom questions, while the class with lecture-discussion 
first failed to answer 58 per cent of the questions put to students by 
the teacher. It appeared that thi lecture-discussion helped students 
In working on the program, and that the program faclllti»ted classroom 
discussion. While most teachers would probably choose the latter advan- 
tage, the relative effectiveness of the two procedures for improving 
criterion test performance remained a question for more study* 



Spring 1962 Classroom Pilot Study 

The two college preparatory classes taking U. S. Government during 
the Spring 1962 semester provided data on the classroom use of all programs 
and on the relative effectiveness of several techniques for combining 
the programed Instruction with standard classroom procedures. 

For the purpose of this tryout. Unit III ("The Structure and Function 
of the Federal Government") was divided into five teaching segments i 



1. Congress and Its Powers 

2. How Congress Makes Laws 

3. The President and His Powers 

4. The National Administration 

5. Our Federal Court System 



Ten methods for combining programing with teacher-oriented Instruction 
^re tried out during a month of classroom Instruction. Two different 
combinations were evaluated during each of the five segments, with the 
two classes receiving different combinations. The methods comparisons 
used may be suoiiiarlsed briefly as follows: 



Segment 1 writ«t^-in programs with .vs. 

follow-up lecture-discussions 
based on students ' critiques 
of the program 



Segment 2 write-in programs with vs. 

follow-up lecture-discussions 
and a multiple-choice program 
for review 



lacture-diacusaions.over the 
regular textbook assignment, 
based In part on students' 
critiques of the previous 
day's lecture 

Introductory lectures with 
write-in programs only 
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Segment 3 



Segment 5 



Introductory lecture days vs. 

followed by multiple-choice 
program days 

write-in programs as home- vs. 

work followed by discussion 
sessions with the teacher 
answering questions, and a 
multiple-choice program as 
review homework 

write-in and multiple-choice vs. 
programs without lecture or 
discussion 



multiple-choice program days 
and follow-up lecture- 
discussion days 

write-in program as homework 
followed by discussion sessions 
with students answering other 
students' questions, and a 
multiple-ctoice program as 
review homework 

the sequence of introductory 
lecture, write-in program, 
follow-up lecture-discussion, 
and multiple-choice program 



A detailed daily lesson plan was prepared for use by the teacher. 

A sunmariaed outline of the lesson plans for the five se^nts of instruction 
is presented below. 



D 

A 

Y 



D 

A 

Y 



D 

A 

Y 

3 



D 

A 

Y 



CLASS ONE 



CLASS TWO 



Segment 1; Congress and Its Powers 



Text Assignment: Chapter 16 

(optional) _ 

WRlTE«^N^ROGR]arbegun in class and 
completed at home, if necessary; 
LESSON CRITIQUE last few minutes of 
class period. 

FOLLOW-UP LECTURE over topics in 
Day 1 program. 



D 

A 

Y 

5 



WRITE-IN FROGRAN in class and com- 
pleted at home if necessary; 
LESSON CRITIQUE last few minutes 
of class period. 

FOLLOW-UP LECTURE over topics in 
Day 3 program. 



EXAcilNATION over materials pre- 
sented Days 1 through 4; 
QUESTIONNAIRE over Sesnent 1; 
Segment 2 WRITE-IN PROGRAM to be 
completed at home. ____ 



Text Assignment: Chapter 16 

L^TURE over topics in first half of 
Pay X program; LESSON CRITIQUE last 
few minutes of class period. 

"lecture over topics in second half 
of Day 1 program; LESSON CRITIQUE 
last few minutes of class period. 



LECTURE over topics in first half of 
Day 3 program; LESSON CRITIQUE last 
few minutes of class period. 

'lECTiSe over topics in second half of 
Day 4 program; LESSON CRITIQUE last 
few minutes of class period. 

EcaMINATXON over materials presented 
Days 1 through 4; QUESTIONNAIRE over 
Segment 1. 



(Lesson plans continued on next page.) 
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CLASS ONE 



Segment 2; 



CLASS TWO 

How Congress Makes Laws 



D 

A 

Y 

1 



Text Assignment: Chapter 17 

(optional) 

First half o£ isiTB-IN PROGRii coif 
pleted at home before class; 
FOLLOH-UP LECTURE over topics in 
program completed at home; second 
half of WRITE-IN PROGRAM begun in 
class and completed at home if 
necessary. 



Text Assignment: Chapter 17 

(optional) 

lMbDUClx>RY LECTURE topics in 
first half of WRITE-IN PROGRAM; WRITE- 
IN PROGRAM begun in class and completed 
at home if necessary. 



D 

A 

Y 

1 

D 

A 

Y 

3 



FOLLOW-UP LECTURE over topics in 
Day 1 program; MULTIPLE-CHOICE 
PROGRAM begun in class and com- 
pleted at home if necessary. 

''points'” to" LOOK FOR "iN FIUl'' * 
SHEETS; FILM, "The Legislative 
Process"; EXAMINATION over the 
materials presented Days 1 and 2. 



INTRODUCTORY LECTURE over topics in 
second half of WRITE-IN PROGRAM; 
WRITE-IN PROGRAM begun in class and 
completed at home if necessary. 

FUM.^'The Le^slative Process"; 
EXAMINATION over materials presented 
Days 1 and 2. 



Segment 3: The President and His Powers 



Text Assignment: Chapter 18 

_ (optional) 



Text Assignment: Chapter 18 

(optional^^ 



D 

A 

Y 

1 

D 

A 

Y 

1 

D 

A 

Y 

1 

D 

A 

Y 

A 



D 

A 

Y 

5 



questionnaire over Segment 2; 
INTRODUCTORY LECTURE over topics in 
Day 2 program. 

FnMT ''The“congress" .HiULTIPLE- 
CHOICE PROGRAM begun in class and 
completed at home if necessary. 

limWDlKJTORY” LECTURE^over topics^ 
in Day 4 program. 



"POINTS TO LOOK FOR IN FIIM" 
SHEETS; FIIM, "The President"; 
MULTIPLE-CHOICE PROGRAM begun in 
class and completed at home if 
necessary. 

EXAMINATION over materials pre- 
sented Days 1 through 4; QUESTION- 
NAIRES over Segment 3; FIIM, 

"T-Men of the treasur^'; Segment 4 
WRITE-IN PROGRAM to be completed 
at home. 



QUESTIONNAIRE over Segment 2; FIIM, 
"The Congress" ; MULTIPLE-CHOICE 
PROGRAM begun in class and completed 
at home if necessary. 

FOIxbW^"" LECTURE^owr topics in 
Day 1 program. 



FIIM, "The President"; MULTIPLE- 
CHOICE PROGRAM begdn in class and com- 
pleted at home if necessary. 

FOLLOIMJP^ LKTimE^over topics in 
Day 3 program. 



EXAMINATION over materials presented 
Days 1 through 4 ; QUESTIONNAIRES over 
Segment 3; FIIM, "T-Men of the 
Treasury" ; Segment 4 WRITE-IN PROGRAM 
to be completed at home. 



(Lesson plans concluded on next page.) 
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CLASS ONE 



CLASS TWO 



Segment 4; The National Administration 



Text Assignment: Chapter 19 

(optional) 



Text Assignment: Chapter 19 

(optional) 



D 

A 

Y 

1 



D 

A 

Y 

2 

7 

A 

Y 



D 

A 

Y 



First half of WRITE-IN PROGRAM com- 
pleted at home; DISCUSSION consis- 
ting of student questions and 
teacher answers over topics In 
program completed at home; WRITE- 
IN PROGRAM to be completed at home. 

DISCUSSION consisting of student 
questions and teacher answers over 
topics In Day 1 program completed 
at home; MULTIPLE-CHOICE PROCStAM 
to be completed at home. 

FOlibfW^UP”" LECTURE over Topics In ~ 
Day 1 and Day 2 programs; MULTIPLE- 
CHOICE PROGRAM to be completed at 
home. 



First half of WRITE-IN PROGRAM completed 
at home; DISCUSSION consisting of stu- 
dent questions and comments and student 
answers over topics In program completed 
at home; WRITE-IN PROGRAM to be com- 
pleted at home. 



DISCUSSION consisting of student ques- 
tions (and comments) and student answers 
over topics In Day 1 program completed 
at home; MULTIPLE-CHOICE PROGRAM to be 
completed at home. 

DISCUSSION consisting of student ques- 
tions (and comments) and student 
answers over topics In Day 1 and Day 2 
programs; MULTIPLE-CHOICE PROGRAM to be 
completed at horse. 



EXAMINATION over material pre- 
sented Days 1 through 3; (^STION- 
NAIRE over Segment 4; FIIM, "The 
Supreme Court." 



EXAMINATION over material presented 
Days 1 through 3; QUESTIONNAIRE over 
Segment 4; FILM, "The Supreme Court." 



Segment 5: Our Federal Court System 



D 

A 

Y 

D 

A 

Y 

A 

D 

A 

Y 



D 

A 

Y 

4 



Text Assignment: Chapter 21 

(optional) 

imifi-IN PROGR^ begim Tn TlaTs 
and completed at home If necessary. 



Text Assignment: Chapter 21 

(^tlonaO _ 

INTRODUCTORY LECTURE over topics In 
Day 1 program; WRITE-IN PROGRAM begun 
In class and conq>leted at home. 



MULTIPLE-CHOICE PROGRAM begun In 
class and completed at home If 
necessary. 

UNlf III EXAMINATIW (PART T) 
EXAMINATION over materials pre- 
sented Days 1 and 2; UNIT III 
EXAMINATION (PART 2). 



FOLLOW-UP LECTURE over topics In 
Day 1 program conq>leted at home; 
MULTIPLE-CHOICE PROGRAM begun In 
class and completed at home. 

UNIT*” III KCAMINATION (PART 1) ; EXAMINA- 
TION over materials presented Days 1 
and 2; UNIT III EXAMINATION (PART 2). 



UNIT III EXAMINATION (PART 3) ; 
QUESTIONNAIRE over Segment 5. 



UNIT III EXAMINATION (PART 3); 
CUESTIONNAIRE over Segment 5. 



Results of segment tests . U. S. History grade-point average and the 
verbal scale of the Differential Aptitude Test (DAT) were used as covariance 
predictor variables In evaluating the segment test results, because the two 
classes differed considerably on these two variables (to the advantage of 
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Class Two) . Table 1 shows the adjusted test score means and estimated 
standard deviations for the two classes on the segaaent tests* 

Table 1 



Adjusted Segment Test Means 



Segment 


Class One 
(NS33) 


Class Two 
(N=32) 


Estimated 
Standard Deviation 
(N*65) 


Maximum 

Possible 

Score 


1 


41*5 


39*6 


5.83 


57 


2 


20*8 


21*6 


2.69 


25 


3 


35.3 


36.0 


4.45 


51 


4 


35.2 


33.9 


3.45 


42 


5 


26.1 


28.1 


2.96 


33 



Effect of lectures . The only difference between means that was 
statistically reliable by the method of covariance was the difference for 
Segment 5 (F«6.59, P<*025 for df»l,61). Class Two, which had the intro- 
ductory and follow-up lectures combined with the programs, performed on 
the test reliably higher than Class One, which had no lectures with the 
programs. The conclusion that combining lecture-discussions with programs 
can improve test performance appears to hold up against some of the argu- 
ments that might qualify or delimit this conclusipn. It might be expected, 
for example, that Class Two spent more time studying the Segment 5 topic 
than Class One, since the class time taken away from working on the 
programs to participate in lecture-discussions would have to be made up 
finishing the programs at home* This was not the case, according to 
the questionnaire data from students. The difference in homework time 
on the programs was compensated for by the fact that Class One spent 
twice as much time studying the textbook as did Class Two. 

Factors underlying the small obtained differences . While not 
statistically reliable, the Segment 1 difference in favor of Class One 
(which had programing) over Class Two (which did not) was of the same 
magnitude as the statistically reliable difference in Segment 5. In 
general, however, the magnitude of test-score differences for the method 
combinations was ouite small* In retrospect, these obtained small 
differences between methods could be attributed to a combination of 
two elements of the conditions under which the methods were evaluated. 

First, tests were given at frequent intervals, ^ich could be expected 
to maintain a high level of motivation* (There is a suspicion that 
write-in programs have most effect, as conq>ared to other study methods, 
when motivation is low* See discussion, Goldbeck and Briggs, 1962.) 

Second, the material was covered at a slow enough pace so that motivated 
students could spend considerable time studying the material at home* 

Such a combination of conditions would not appear to be optimum for 
comparing teaching methods, since high amounts of independent (uncontrolled) 
study would tend to minimise the measurable effects of controlled learning 
conditions. The implication for the conduct of subsequent research 
was obvious* 
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An aimlyslt of effect! of the multlple^cholce progmat . The failure 
to find a teat performance advanti^ge for Claaa One on Segment 2 uaa aome- 
what aurprlaing, alnce thla claaa waa given both write-in and multiple- 
chok^e programa while Claaa Two waa given only the write-in program. The 
obaerved teat mean difference actually favored the claaa receiving only 
the write-in program. Although the aequence of lecture and program 
differed in theae two claaaea during Segment 2, the reaulta of Segment 3 
ahowed that the aequence of program and lecture produced no appreciable 
effecta on teat performance. The apparent implication waa that the 
multiple-choice progr« " Aid not add appreciably to teaching effectiveneaa . 

Thla Implication received aome confirmation from an examination of the 
percentage-correct acorep for the five aegmenta, corrected for chance by 
the following formula: 

T" - chance acore 

P* » X 100 

maximum acore - chance acore 

where T' la the mean teat acore adjuated by the DAT verbal acale 

and the U. S. Hiatory grade-point average covariance predictor variables. 

These percentage acorea are shown in Table 2. 

Table 2 



Seggient Percentage-Correct Scores 



Segnent 


Class 1 


Clasb 2 


1 


63 


58 


2 


75 


79 


3 


55 


57 


4 


78 


73 


5 


70 


79 



While there is no good basis for assuring that the segment tests were 
of equivalent difficulty, it nevertheless was the case that the lowest 
scores were obtained for Segment 3, the only segment where the multiple- 
choice program was the only program used. 

The only indication that the multiple-choice programs were as 
effective as the write-in programs came from the students in their replies 
to a questionnaire. Class One showed a slight preference for the write-in 
program at the end of Segment 2 (by a 13:11 ratio). At the end of Segment 4, 
when both classes were asked for a preference, Class One showed a shift in 
preference to the multiple-choice programs with nearly twice as many students 
preferring the multiple-choice program. Students in Class Two were divided 
nearly equally in their preference. Although the questionnaire asked 
students to indicate both the program "most easy to use" and the program 
from which they "learned the most," and although these two forms of the 
question showed nearly equal indications of preference, it is likely that 
students' opinions of which program taught best were influenced by the 
consideration of amount-of-work-required in completing the program. The 
fewer number of responses required by the multiple-choice program, along 
with the ease of finding these responses in the feedback paragraph. 
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certainly made the multiple-choice program easier to fill out than the 
write-in program. More evidence on the relative superiority of the write-in 
programs used will be found in the analysis to follow. 

Relative superiority of the write-in programs . In order to assess 
the relative contribution of multiple-choice programs 9 write-in programs 9 
and lectures to criterion performance, a detailed analysis was performed 
on the content coverage of these media in relation to criterion test items. 

(In the case of the lectures, this analysis was based on transcribed tape 
recordings of all classroom sessions.) 

Four categories were established to describe how the subject-matter 
material covered by criterion tests was presented to the students: 
directly-stated (DS) ; with-all-essential-information (W) ; without-all- 
essential- information (WO) ; not-mentioned (NM) . Under the first category (DS) , 
material needed to answer criterion test items was presented by programs or 
lectures in a direct-statement fashion; that is, in the programs or lectures, 
students heard or read material which subsequently appeared on a test in 
nearly identical form. Under the second category (W), knowledge required 
for successful criterion performance was simply discussed fully so that, 
although a criterion test question and answer was not presented ’’word for 
word" by the programs or lectures, all essential inforsiation students would 
need to know in order to correctly answer a test item was thor oughly covered 
in the course of topic discussion. Under the third category (WO), program 
or lecture discussion of topics covered by test items failed to present all 
essential information students would need to know in order to correctly 
answer the test item. Finally, under the fourth category (MH), material 
pertinent to test items had been omitted entirely from the programs or 
lectures. 

The first two categories, directly-stated (DS) and with-all-essential- 
elements (W), were designated as "adequate" handling of this material; the 
third and fourth categories, without-all-essential-information (WO) and 
not-mentioned (NM) , were designated as "inadequate " handling of criterion 
test material. Analyses of these DS, W, WO, and m categories were utilised 
in determining the existence and nature of interactions among the three 
major instructional media when used in different combinations. 

Assigning one of the four categories (DS, W, WO, or NM) to lecture and 
program treatments of subject-matter was accomplished in the following way. 

For each item on a criterion test, the lecture transcripts were examined 
until a discussion of the material covered by that test item was located. 

Based upon how adequately the information needed to correctly answer the 
test item was presented in the context of the lecture-discussion, an 
assignment of either the DS, W, or WO categories was made to lecture 
treatment of material covered by the test item concerned. If the topic 
was found to have been omitted entirely from lecture-discussions, an 
assignment of the IM category was made to lecture treatment of material 
covered by that test item. A similar search was made of multiple- choice 
and write-in programs to locate the discussion of material covered by the 
same test item, and again, one of the four categories was assigned to 
multiple-choice and write-in program treatment of the material covered 
by that test item. The above procedure was repeated until every item on 
all the criterion tests had been "keyed" to the lectures, the multiple- 
choice programs, and the write-in programs in terms of the DS, W, WO, or 
til categories. 
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Next, the percentage of students who knew the correct answer (as 
opposed to selecting the right answer by chance) to the individual test 
item was determined to evaluate the effect of lectures and programs on 
criterion performance. In order to estimate the percentage of students 
who scored correctly on a test item through correct knowledge or correct 
reasoning (P^). rather than through guessing , the raw percentage of 
students succeeding on each test item was corrected for chance. (See 
formula for computing on page 6.) Although the corrected value for 
the percentage of students knowing the answer to a particular item is 
only an approximation, this corrected percentage can be expected to 
yield a better identification of what students actually have learned 
than the raw percentage of students succeeding on that item. 

The discussion of the findings from these data will be concerned 
with comparing the three major instructional media (multiple-choice 
program, write-in program, and lecture) and the four discussion condi- 
tions (DS, W, WO, and Hi) in terms of their relative "effectiveness," 
where the term "effectiveness" refers primarily to the magnitude of 
effect any condition exerts on criterion test performance. 

A comparison of test performance in terms of weighted mean P^'s 
showed that test performance increased- 13 per cent for test items 
adequately covered (DS or W) in the programs (over items not adequately 
covered in the programs), but increased only 6 per cent for items 
adequately covered in the lectures (over items not adequately covered in 
the lectures). Within this descriptive framework, the programs had more 
effect on test performance than did the lectures. It is relevant to 
report that both types of programs (multiple- choice and write-in) pre- 
sented over four-fifths of the iiiaterial covered on tests adequately 
(multiple-ch'>ice programs « 54 per cent DS, 33 per cent W; write-in 
program ■ 43 per cent DS, 39 per cent W). On the other hand, lectures 
in both classes presented one-third of the material adequately and 
omitted over half of the material - covered by the criterion tests 
(Class One lectures ■ 55 per cent NM; Class Two lectures * 54 per cent HI). 

The degree of adequacy (DS, W, NM) of program coverage of test items 
had more effect on test performance for items which had been well covered 
in lectures (DS and W) than for items not mentioned in the lectures. That 
is, for items categorised as DS in the lectures, adequacy of program 
coverage was associated with an incr^nent in value of 35 per cent 
(from 50 per cent for Hi program coverage to 85 per cent for DS program 
coverage). For items categorized as W in the lectures, adequacy of program 
coverage was associated with an increment in P^ value of 17 per cent (from 
60 per cent for Hi program coverage to 77 per c ent for DS program coverage) . 
The corresponding increment for test material omitted entirely (Hi) from 
lectures was 9 per cent (from 65 per cent for Hi program coverage to 
74 per cent for DS program coverage). 

A high degree of lecture coverage of test items had a facilitating 
effect on test performance for items which had been directly stated in 
programs (DS), but had a negative effect for items not mentioned in the 
programs. For example, for items categorised as DS in the programs, 
adequacy of lecture coverage was associated with an increment in value 
of 11 per cent (from 77 pert;cent for Hi lecture coverage to 88 per cent 
for DS lecture coverage). For W program items, adequacy of lecture 
coverage was associated with an increment of only 2 per cent (from 
69 per cent for Hi lecture coverage to 71 per cent for DS lecture coverage) . 
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For test items not mentioned in the progrsms (Hi), adequacy of lecture 
coverage was associated with a decrement in value of 18 per cent 
(from 68 per cent for W. lecture coverage to 5O per cent for DS lecture 
coverage) • 

There are several apparent generalisations that may be drawn from the 
above findings . First, programing and teacher presentations were mutually 
facilitating under the conditions of instruction used . This outcome should 
suggest a bit of humility for both programers and lecturers who advocate 
exclusive use of their specialty. Further, it indicates that the combina- 
tion of these instructional media may require a careful balance of content 
coverage as well as the usual consideration of careful preparation . While 
it might be argued that special emphasis on careful preparation of a single 
n^dia for independent use may obviate the necessity to consider the integra- 
tion of two or more instructional media, the latter alternative provides a 
promising avenue for the development of improved teaching procedures. This 
alternative of ccnnbined media has not always been fully recognized, and the 
advent of programing as a useful and convenient instructional medium makes 
this alternative especially attractive. The exceptionally strong dependency 
found for the lecture-discussion medium on programing coverage raises a 
challenge for the teacher, along with the promise for improved teaching 
procedures • It is indicated that the teacher must consider choosing wisely 
the topics which go beyond the scope of the program and must exercise skill 
in the wav that they are introduced. A similar challenge is put to the 
programer who must prepare programs which provide students with the kind of 
learning that enables the teacher to present the kind of material he believes 
is most appropriate and instructive . 

Another finding from these data showed that the write-in program had 
more overall effect on test performance than did the multiple-choice program. 
That is, idien test-performance curves for both classes were combined, it 
was found that test performance increase 13 per cent in value for test 
items adequately covered by the write-in program (DS and W) , over items not 
adequately covered by the write-in program (NM), On the other hand, test 
performance showed almost no increase in P^. value (1 per cent) for test 
items adequately covered by the multiple-choice program , over items not 
adequately covered by the multiple-choice program. This is in line with 
the previously discussed finding that the multiple-choice program was not 
as effective a teaching medium as the write-in program. 

Several qualifying factors may be considered here. First, when 
write-in and multiple-choice programs were used in combination within a 
segment, the write-in program was always administered first. Under these 
circumstances, the write-in program may have served to accomplish original 
learning, and the multiple- choice program may have essentially served to 
highlight and review main points. (Or, this may be a case illustrating 
Jost's law pertaining to associations of equal strength but unequal age. 

At any rate, an independent study (Campbell, 1962) of branching programs 
suggests that if an item of information is not grasped at first presentation, 
it takes considerable repetition or new exposure to establish it firmly.) 

A related consideration is that students may have built-in criteria 
as to what they think they should learn, and how well they should learn it. 
To a certain extent, these criteria have been "built in" by students' 
previous classroom experience. For instance, if students felt the write- 
in program contained "all they should know," they may have completed the 
multiple-choice program more perfunctorily. 
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Second, the write-in program required students to make more responses 
than did the multiple-choice program, and the greater number of write-in 
responses may have required a greater amount of rereading of the write-in 
program. In contrast, the multiple-choice program had feedback printed 
on the following page, which made it relatively easy for students to find 
correct answers without rereading previous material. Student-completed 
questionnaires confirm that the write-in programs took much more time to 
complete than the multiple-choice programs due to the necessity to 
( 1 ) reread previous material to find answers, instead of turning to a 
following page for feedback, and (2) write out the answer instead of 
check-marking an alternative. If it is true that the extra time spent 
rereading material and writing out answers helps students to learn better ' 
than the multiple-choice procedure, it is reasonable to expect the write-in 
programs to affect test performance to a greater degree than the multiple- 
choice programs. 

These findings , coupled with the earlier findings which indicated 
a relative weakness of the multiple- choice programs compared to the write-in 
programs, led to the conclusion that for the purposes of the Fall 1962 

test^ inclusion of the multiple-choice programs would not be warranted^ 



Phase III; Fall 1 9^2 Field Test of the Composite Instructional Methods 

The major purpose of the field test held in the Fall 1962 semester 
was __to introduce programing under classroom conditions in theoptimum 
combin ation with teacher-oriented instruction , as determined by the 
preliminary and classroom-pilot studies conducted under this project. 

The evaluation of this field test could have been based solely on 
the administrative success of presenting an integrated combination of 
programing and teacher-oriented instruction. For although both instruc- 
tional media had been shown to be effective teaching procedures and to be 
mutually facilitating, there remained potentially difficult problems in 
scheduling the presentation of these media efficiently in an effective 
combination. 

A possible approach to scheduling integrated programed instruction 
would be one which took advantage of the programing feature to allow 
students to progress at their own rates. To combine teacher instruction 
with programing used in this manner, however, would seem to require 
solution to the problems of flexible scheduling. That is, groups of 
students who progress through the programs at the same rate could be 
gathered together for teacher-conducted sessions at appropriate points 
in the curriculum. But until classroom facilities are augmented or 
redesigned to accommodate this type of procedure, it would not seem 
feasible to plan any large-scale introduction of programing within this 
conceptual framework. 

The more practical goal at the present time would seem to be a 
procedure for introducing programing compatible with the current 
classroom conditions and school scheduling constraints. On the Fall 1962 
field test, therefore, students were all assigned the same portion of 
programing each day and all participated in lecture-discussions in 
normal class groups. 
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Since the preeentation of program material and lecture-diacuaiion 
material waa aynchronised with reapect to rate of topic coverage, con- 
aiderable emphaaia waa given to requiring atudenta to complete each day 'a 
progrim aaaignment* The introductory remarka (aee Appendix A) read to 
the atudenta on the firat day of the field teat reflect thia emphaaia. . 
Alao, a Program Aaaignnient Schedule (aee Appendix C for copiea of the 
achedulea given to atudenta and to teachera, or ”Non-Sa*’) maa given to 
each atudent with the onderatanding that in the event of any abaence, die 
programa aaaigned would be due on the day of return to claaa. (Programa 
were delivered to abaenteea by their claaamatea on a volunteer baaia.) 

In order that the teacher could accurately monitor the completion of ^ 
programa, an Attendance and Aaaignment Control Sheet (aee .Appendix * 

waa paed to record the date on which each atudent received and completed 
each of the programa. Thia device waa eapecially uaeful in keeping 
account of abaent atudenta and in demonatrating to all atudenta that 
failure to complete a program on time would be noted and recorded. 

The feature of programa which requirea written reaponaea providea 
overt indication when an aaaignment haa been completed. Thia feature, 
coupled with the uae of the Aaaignment and Control Sheet to emphaaiee 
the importance of completing aaaignmenta, haa advantagea over the text- 
book for the efforta of both atudent and teacher in monitoring progreaa. 

A daily leaaon plan (aee Appendix D) waa prepared for the four weeka 
of inatruction covered by the field teat. Thia leaaon plan waa bP 
the teachera aa a guide for integrating their lecture-diacuaaion with 

the programing. 

The five teaching ae^nenta uaed in the Spring 1962 claaaroom 
atudy were collapaed to form three aegm^nta, with a teat (aee Appendix H) 
after each segment, aa followa: 

Segment 1 : Congreaa 

Segment 2: The Preaident and the National Adminiatration 

Segment 3: Our Federal Court Syatem 

Thia achedule reduced to a minimum the number of teaching and atudy 
and at the aame time increaaed the ratio of teaching daya to teating daya, 
in order that the effectiveneaa of inatruction could be evaluated with 
minimum opportunity for independent (uncontrolled) atudy time to confound 
the effects of the instructional method. 

Since the Spring 1962 classroom pilot study indicated that both 
introducS:s*y and £ollow*up lecture-discussions %iere effective adjuncts 
to the programs, both sequences of lecture-discussions were integrated 
with the programa. Also, aa in the previous pilot study, students were 
given some time to work on the programs in class, and were requirea to 
spend homework time in completing the programs. Since students differ 
in their rates of completing the programs, it was arranged that the first 
part of the class session be’ devoted- to lecture-discussion, with the 
latter part of the class session allotted to starting work on the program. 
Students could thereby compensate for different work rates in class by 
spending different amounts of time working on the programs at home. 

All students would then come to class the following day having covered 
the same program material . 
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Additional advantages of this form of icheduling were that (1) the 
teacher had classroom time to complete record-keeping work and to begin 
preparation for subsequent class work, and (2) the students made a start 
on their homework assignment. (During the Spring 1962 classroom pilot 
study, students reported that having time to make however small a start 
in class on the homework assignment enhanced motivation to complete the 
assignment— an effect similar to the Zeigamik phenomenon reported by 
experimental psychologists)* Part of the teachers' classroom preparation 
time was devoted to reviewing the lesson critiques (see Appendix G) 
completed by the students after each program. This lesson critique 
was used by the teacher to guide his presentation for the review session 
preceding each test day. 

The write-in programs used in the Fall 1962 field test (see Appendix F 
for a sample portion of the program) had undergone final revision based on 
the data collected during the Spring 1962 classroom pilot study. The 
multiple-choice programs were not used in the final field test because 
data from the Spring 1962 tryout indicated that their use would not 
contribute appreciably to teaching effectiveness. 

In addition to demonstratine how programed instruction co uld be 
integrated efficiently with conventional classroom teachi ng, the Fall 1961 
field test also was intended to compare the success of such an integrated 
instructional system with a conventional system of inst ruction, as 
measured by performance criterion tests. 

Since there is a prevalent trend toward grouping students in classes 
according to some ability and/or achievement classiflcat'^'n, and since 
the U. S. Government course used in the present study followed this 
practice, the efficacy of integrated programed instruction was evaluated 
with two levels of students (college preparatory and general levels) . 

In order to obtain the mr/St information about programing, teacher, and 
student variables (^using the two available teachers, each of whom taught 
three classes), conditions were assigned to classes as shown by the *'x*s" 
in the following table: 





Teacher A 


Teacher B 




College Preparatory 


General 


College preparatory 


General 


Programing 


X 


X 


X 


X 


Non-Programipg 


X 






X 



This arrangement made it possible to compare (1) programing vs. non-programing 
for both college preparatory and general classes, and (2) teacher differences 
in introducing programing for both college preparatory and general classes. 

Before discussing the results of the field test, it is relevant to 
describe a change made in the lesson plan immediately prior to the presenta- 
tion of Segment 3, *’Our Federal Court System." This change consisted of 
asking the teachers to assign the textbook chapter (which essentially 
paralleled program coverage of topics in this segment) as required reading 
for the four programing classes, in addition to completion of the program 
on the Federal courts system. This innovation was introduced because 
continuing in Se^nent 3 the integrated programing conditions used in the 
first two segments was not expected to show any new trends or results • 
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This expectation vas founded on two outconee of an analyele of the reeulte 
of Segpients 1 and 2. This analysis was completed Just in tisie to initiate 
the change in procedure referred to inmediately above, relating to assign- 
ment of textbook reading for Segment 3. This initial analysis revealed 
that (1) the field-test results from these first two segments were clear cut 
concerning the superior performance of programing classes compared to 
non-programing classes, as measured by the end-of-segpent tests, and 
(2) the results from these two segments were consistent, in that the 
magnitude of differences between programing and non-programing classes 
were essentially the same— also, no teacher differences in introducing 
programing to general and college preparatory classes were found for 
either of the two segpients (see Table 3). Therefore, Segment 3 presented 
an appropriate opportunity to integrate programed instruction with still 
another conventional teaching medium (reading of the textbook) . 



Results 



Table 3 shows the field-test N's, means, and standard deviations of 
the three segment tests and the DAT verbal scale means for the six classes 
participating in the study. Also shown are the values for Segments 1 and 2 
combined, since the instructional method conditions were equivalent for 
these segments and the independent- variable effects on test-score results 
for them were strikingly similar. 



Table 3 

Field Test N*s. Means, and Standard Deviations 











Segment Tests 




DAT(v) 


I 


1 


11 




1 and II 
Combined 


III 


Classes 


N 


Mean 


SD 


|N 


Mean 


SD 


N 


Mean 


SD 


N 


Mean 


SD 


N 


Mean 


SD 


GENERAL LEVEL 
































Program 
Teacher A 


22 


42.9 


24.42 


22 


48.6 


7.68 


22 


53.7 


9.29 


22 


102.3 


15.79 


22 


29.6 


3.81 


Progrim 
Teacher B 


22 


53.2 


26.17 


22 


48.3 


7.56 


22 


55.0 


10.17 


22 


103.3 


16.41 


22 


30.5 


3.88 


Non-Program 
Teacher B 


28 


46.0 


23.02 


28 


39.0 


9.77 


28 


44.5 


11.93 


28 


83.5 


20.55 


28 


27.1 


6.33 


COLLEGE PREP. 
LEVEL 
































Program 
Teacher A 


28 


77.9 


19.49 


28 


56.5 


6.67 


28 


62.9 


7.36 


28 


119.4 


13.03 


28 


31.8 


4.38 


Program 
Teacher B 


25 


78.0 


20.03 


25 


56.6 


6.89 


25 


62.6 


9.19 


25 


119.2 


15.33 


25 


35.3 


4.72 


Non-Program 
Teacher A 


25 


76.8 


23.02 


25 


49.7 


7.91 


25 


55.0 


9.83 


25 


104.7 


16.71 


25 


33.9 


4.85 
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It should be noted that although there vaa no disparity in D« verbal 
scores anong the general-level classes (F=1.61, P>.05 for df»2,79), there 
was an inordinate nuaher of extreme scores at the low of tte 
programing class distribution. Therefore, it ««• 

classroom data to omit from the test-score evaluations all students with 
a DAT yarbal percentile score of less than tens Eight students were 
omitted from Teacher B*s general- level, non-programing class, and two 
students were omitted from Teacher B’s general- level, programing class. 
This selective sampling resulted in more comparable DAT means for the 
general classes (F-1.03). 

Abaen ce of teacher differences for S egnents 1 and 2. Examining first 

the field-test data from Segments 1 and 2, it is *PP**®"* 
showed no differences between the two teacters in presenting t^ ij^fred 
programed instruction. Test means reflecting the teacher effect differ, 
only sllghUy> these differences were not statistically reliable. 

Simerlorltv of Integrated nm eramlng classes (Segments 1 and _2) . All 
the in^grated programing cUsses perform ed ^®»e» ""J**® * 

tests than did the non-programing classes. Difference s 
integrated orogramlng on the combined Segment 1 and Segment 2 tests were 
StJcXtSfe for the students of both the c«,ll^e 
cUsses (F.16.30, P<.01 for df«l,76) and the general- level classes 
(F-19.94, F< .01 for dfn,70). The clear-cut and 

of integrated programing in raising test perfo^nce de^ adolescent 

this teaching approach is far enough ***®. !f^!Lcatio^ 

period to be considered seriously for the full-time 
This finding is especially striking because only ® 
time was devoted to programed learning. As a ^u^fther ^Jt 
students averaged from three to four minutes of 

page of progra^ng. (A page in these programs, it should be t eme^re , 
presents much more material than does a "frame" in most programs.) 

Effectiveness of integrated p rograming by ability level 
and 2) . As for the question of whether the S*“!!SLry level 

more effective in improving performance ** fnsicste that the 

or the general level, data in Table 3 would seem 
general-level classes were more amenable to the P”^*"f;“* 

It is difficult to be definite on this point from the present tota, 
however, due to the fact that in the general-level ®J®“®®* 

U. S. history grade-point average was substantially i.. 

non-programing class than for the j- ooint average 

of siud^ts. matched on the basis of U. S. 

were drawn from the programing and non-programing clMses t a g ^ y 

the smse teacher (H.14). the mean scores on Jh® ®“»>toed 

Segment 2 tests were 103.2 and 89.3, MM«atorv 

parable to ptogranrfng/non-programing differences faanA for cdUe^ w. ^ 

c“sse‘. TheVssiblllty rmnalns. of course, that P««J^ 
prepared to be more suitable for either higjh- or low “ 

For classes of exceptionally high- or low-ability •*“ * . ’ „ ^ 

possibility should be given serious consideration. 

imnlication of the results fro m the present study, however, pp , — 

i-h.i - M.» programing, u sed under the same conditions, may , be 

effective for both high- and low-ability groups. 

(Aside from the present study, independent evidence supports the 
above supposition that for maximum effectiveness programs may need 



to vttTy According to ccrtAln chcrActeristics of the group for vhich.thpy 
are Intended. An analysis just completed by Wayne Hershberger, under a 
contract sponsored by the Office of Naval Research, Contract No. Nonr- 
3077(00), siipports the hypothesis that linear ’’small-step" programing 
helps students in Inverse proportion to their reading ability, while a 
certain form of large-step, or "reading," programing helps students 
in proportion to their reading ability. It should be noted, however, 
that the linear programs utilised in the present study are not taken 
as typical of either "large-step" or "small-step" programs in general. 
Nevertheless, continued research on these matters may reveal ways to 
optimally adjust programs to individual and class differences.) 



Shorter test-completion times for programing classes ( Segment 2). 
During the Segment 1 test, it was noticed that th^ programing classes 
seemed to complete the test more quickly than the non-programing classes. 
Therefore, test-completion times were recorded for all students on the 
Segment 2 test. Mean time to complete the test for both general- level 
and college preparatory programing students was 27 minutes. Mean test 
completion times for the general-level and college preparatory non- 
programing students were 31 and 33 minutes, respectively. The 8ho|teg 
test times of the programing students were appar e ntly indicative of 
increased facility in dealing with the cou rse material that was afforded 
by the programing . 



Extra-curricular use of programs by studen ts in non-programii^ 
classes (Segment 3). The data from Segment 3, although based on a 
relatively short period of instruction, provide additional information 
on several points. These include (a) extra-curricular use of programs 
and (b) effects of textbook assignment, as made by the two teachers. 



Many of the students in the non-programing classes had discovered 
that programing was an effective way of improving test performance an 
had begun to use the programs of their friends in the programing classes* 
This practice was documented in part by the questionnaires complete . y. 
these students (see Appendices 1 and J for the questionnaires and summar es 
of student responses), and by discussions with the students which reveale 
that the extra-curricular use of programs during Segment 3 was ^re 
prevalent than the questionnaires indicated. While this form of 
acceptance of the programs was indicative of their utility, it neve 
detracted from the capability to measure this utility by the test results 

from Segment 3. 



While some of the students in non-programing classes were beginning 
to seek out the programs during Segment 3 as a means for i^roving 
their grades, one of the teachers decided that two of his tord-working 
low-ability students in the non-programing general- level class 
be able to pass a test by using the programing, whereas they had been 
unsuccessful before. The DAT verbal scores for these two students were 
in the vicinity of the tenth percentile. Although for other students the 
program error rates were generally low (averaging 2.0 per cent for a 
college preparatory class and 6.3 per cent for a general class), one 
of these two students was exceptionally unable to respond adeq^tely 
on the program. This student's Segment 3 test score was at a failing 
level . The other student was able to respond on the progrw well within 
the program-error-rate range for general-level students. The Seg^nt 3 
test score for this student was a hi^ “C"--the first time she had 
achieved a passing test score in the course. 
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The experience, Just described, with the two students suggests 
two relevant obeservatlons . First, programs prepared for normal group 
use cannot be expected to replace all diagnostic and remedial work for 
special problem cases. Second, were another field test to follow this 
one, all students could benefit from the experimental programing, 
since the superiority of the integrated program groups now has been 
determined. Such a follow-on study, then, could concentrate on some 
other feature, such as devising and testing separate programs for 
general and college preparatory groups. 

Effect of textbook reading and teacher differences ^Segment 3 ) . 
Results from Segoient 3 shed some light on the role of textbook assign- 
ments as a teaching-method component . As Indicated earlier, this was 
the only segment in the field test in which programing classes were 
assigned the pertinent textbook chapter as required reading, in addition 
to completing a program. It Is relevant to note that one of the two 
teachers was more vigorous in making the textbook-reading assignment. 
This teacher variation is reflected in Table 4 by differences between 
Teacher A's classes and Thacher B's classes in the percentage of 
students who reported having read the assigned textbook chapter. 

Table 4 



Percentage of Students Reading Textbook Chapter 

for Segment 3 





Teacher A 


Teacher B 


General Programing Class 


27% 


42% 


College Preparatory Programing 
Classes 


17% 


76% 

! 



These differences between Teacher A's classes and Teacher B's classes 
in reports of reading the textbook assignment were in turn reflected by 
differences between these classes in Segment 3 test-performance scores 
(see Table 3) . The most divergent percentages of students reading the 
textbook were for the two college preparatory classes taught by the 
two different* teachers; it is for these classes that we encounter 
the only occasion of a statistically reliable test-score difference 
associated with teacher difference (F«7.98, P< .01 for dfsl,51), with 
Teacher B's classes performing higher than Teacher A s classes. 

The apparent implications of this finding are (1) textbook reading 
assignments can contribute favorably to the program- lecture combination 
and (2) the effectiveness of the textbook assignment may be dependant 
on the manner in which the teacher makes the assignment. Most readers 
will isnediately think of many other variables that are likely to 
determine the effectiveness of a reading assignment. It is just this 
variability, or unknown resultants, of a reading assignment which points 
up one of the major advantages of programing. The overt indications 
of student responses throughout the program both allow the teacher to 
exercise control more easily over student performance of the assignments 
and provide feedback to the teacher on completion of the assignments, 
thereby assuring that independent study by students is kept current 
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with group work in the classroom (eg., lectures and discussions). Even 
with programs, of course, teachers still need to be concerned that students 
complete their homework assignments. However, the differences associated 
with textbook assignments and the lack of differences associated with 
programing assignments found In the present study are consistent with 
the expectation that programing should alleviate some of the problems 
Involved In Infltsenclng students to study. 

For the general-level programing classes, there were less disparate 
percentages of students who read the Segment 3 textbook chapter assignment 
(see Table 4) . This more comparable amount of textbook reading was 
associated with equivalent test scores for these two programing classes 
(see Table 3); therefore, these two general-level programing classes 
could be combined for comparison with the general- level non-programing 
class. This comparison showed a statistically reliable difference in 
favor of the programing classes (F*5.94, F< .05 for df*l,70). It would 
be Interesting to determine if a larger difference in textbook reading 
for general- level programing classes would result in a greater magnitude 
of test score differences, as was found for the college preparatory 
classes. 

In s*«™ftrv. It should be noted carefully that no teacher diffe rences 
were found regarding ability to utilize the inte grated programed Instruc- 
tion; the only difference, as reported above, resulted from varying 
degrees of emphasis upon assignment of the textbook reading. 



Summary of Final Field Test Results and Th eir Implications 

In summary, the criterion tests showed that the programing, as 
combined with standard teaching procedures in the present study, substan- 
tially Increased the amount of learning compared to the standard teaching 
procedures alone. This facilitation applied to both general-level and 
college preparatory students, and It did not differ for the two teachers 
who conducted the classes during the study. 

There Is some additional information from the field test that 
should be helpful in developing further applications of programed 
learning in the classroom. The reader may find Interesting and valuable 
data from the questionnaires in Appendices I and J . One conclusion 
from these data Is of special and general interest: A preponderance 

of students were enthusiastic about using the programs; most significant 
was the Indication that students came to like the programing more 
as they had more experience with it. There Is always the possibility 
(even with a month-long study) that it is the novelty of the new 
teaching method which produces, or at least influences, the favorable 
results. It might be mentioned parenthetically that any method that 
will Improve student learning for as long as even one month is not 
without value. Nevertheless, it Is in long-term effectiveness that 
we must be most Interested. The report from studen ts that their 
favorable attitude toward programing Increased durin g the month, 
coupled with the lack of anv Indication that the effe ctiveness for 
Increasing criterion test performance tended to drop off , was taken 
as conflr^tlon of the belief that the phenomenon Is not a traps lent, one. 
It Is likely that the success on tests experienced by the programing 
students was itself a partial sustalner of continued effectiveness 
of the teaching method. 
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j — ^ fhm field test for introducing prograning 

The procedure used in tte fieW test i i„,truction aey 

M a ®“*^***^’^ P iMW-ocile Ippllcatione of this approach for 
serve as a model for i^ger sea w hard-and-fast rules for 

Improving education. It is Ihis point is coamen- 

teaching with programing would be appropriate, inw^ ^ 

surate with a general fit his evaluation of the 

progrmsing in the pJobleL he perceives, and 

peculiarities associated with the teachi^ P~t>i^ P 

his own capabilities. (An throu^ the 

of the teachers In the teaching a remedial-level 

prograait as a class exercise ^ . . ,v,nuii be offered for use 

iuis.) certainly, a new ‘“'"ing medta »^",t ad^ntage. Without 

with guidelines as to how it Mwerted to misuse or relegated 

such guidelines, the medius may be P«^***“ “ i„teerated programing 
to pelfunction. Hopefully, the “P"^*** 
will proceed with continued Interaction among teacners. « 

and researchers. 
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APPmXDC A 



imorUCnON to PROCa^AMEiD unit of INSTRtmOW 

(The followlDg is the content ocf Introductory cooments made by the tecushers 
to their FROGBAM classes prior to beginning instruction on THE FEIERAL 

goverhmbntO 



INTRODUCTION TO PROGR/iMBD UNIT OP INSTRUCTION 



During the next few days^ instruction on THE FEDERAL GOVERNMEOT will he 
based on "programed text" instead of the regular texthooh. These 
"programs," as they are called, will he similar to the programs on the 
Constitution which you filled out a few weeks ago. You will usually 
have some time to work on thcj programs in class on the day they are handed 
out. They must he completed (all of the blanks filled in, while you read) 
and returned on the next day of class, at which time the topics covered in 
the programs will be discussed in class. Failure to hand in any program 
on time will result in an "P" grade (equivalent to an examinaticn) for the 
assignment. Additional 'F" grades will be given each day, until the late 
program is handed in. 

You will be given a program assignment schedule that shows when each of 
the programs is assigned and when it is due. You are responsible for 
arranging to have someone in class bring you the assigned program on any 
day that you are absent from class. Programs that have been assigned 
during any absence are always due on the day of your return to class from 
the absence. 

When you work on a program in class, always draw a line across the page 
at the point where you reached before leaving the classroom. This will 
enable us to find out the average amount of time that is nedded to complete 
the programs. We don't want to find out how fast you can complete the 
programs (and you won 't be graded on speed) , but how much time we can 
expect other groups to spend on them. Therefore, we won't be looking at 
how much you finish in class individually, but at the average for the whole 
group. 

While the programs are being used, you will not be required to bring the 
textbook to class. The textbook may be used as an outside reading source, 
Just as the reading of newspapers and magazines is encouraged. 



Hand out Program Assignment Schedules. 



mmmmm 



1 



APPENDIX B 



EXPLANATORY COMMENTS FOR NON-PROGRAM CLASSES ("Period 4") 

(The following Is the content of Introductory comments made by the teachers 
to their NON-PROGRAM classes prior to beginning instruction on THE FEDERAL 
QOVBRHHBDT.) 



EXPLAIUVTOBY COMMENTS FOR PERIOD 4 CLASSES 



During the next few weeks ^ Instruction in some of the other 
classes will be based on ”prograaed text” instead of the 
regular textbook. The prograaed text is a different way of 
presenting the same Information found In your text. It Is 
similar to a workbook In that the student fills in blanks as 
he reads. 

The class will continue to use the text as usual. You will 
be given an assignment schedule that shows when each chapter 
is assigned for reading this month, and when you must complete 
each reading assignment. 

Hand out Assignment Schedules. 



Mm 









APPENDIX C 



PROGRAM ASSIGiaMENT SCHEDULE 

(OJhe following is the PROGRAM ASSIGNMENT SCHEDULE for the PROGRAM classes 
one form, also indicating films and lecture coverage, was given to the 
teachers; the other form was given to students.) 



ASSIGNMENT SCHEDULE 

(The following is the ASSIGNMENT SCHEDULE for the NON-PROGRAM classes; 
one form, also indicating films, was given to the teachers; the other 
form was given to students.) 



PROGHAM ASSIGNMENT SCiIeBULE 
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PROGRAM ASSIGNMENT SCHEDULE (with additional information for »'non-Ss'» 
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AFFESm D 



laSSON PLAHS FOR FROGRAM CLASSES ("Feriods 1, Z, 3, 5”) 

(The following asre cooplete dally leseson plans used hy teachers for the 
FBOCSAM classes.) ' 
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Friday 

2 November DAY 1 LESSON PLAN FOR PERIODS 1, 2, 3, and 5 

1. Introductory reinarks on program-based instruction. 

2. Hand out Program Assignment Schedule. 

3. Introductory lecture ( about 15 minutes) on topics in three programs on 
Congress ; 

CONGRESS IS THE LAW-MAKING BRANCH OF OUR FEDERAL GOVERNMENT 

POWERS OF CONGRESS 

THE HOUSE OF REPRESENTATIVES 

Qualifications^ Term, and Salary of Representatives 
Membership 

Reapportionment and Redistricting 
Privileges of Congressmen 
Sole Powers of the House 
Strong and Weak Points of the House 

THE SENATE 
Membership 

Qualifications! Term^ and Saleu^y of Senators 

Sole Powers of the Senate 

The Senate in Our Legislative System 

SESSIONS OF CONGRESS 



ORGANIZING CONGRESS 

Organization of the House 
Committees of the House 
Party Organization in Congress 
Organization of the Senate 
Joint Standing Committees 
Legislative Reorganization Act 






PING A RECORD PROCEEDINGS 



HOW A BILL BECOMES A LAW 

Introducing a Bill 

Procedure in the House 

Procedure in the Senate 

The Conference Committee 

Log-Rolling 

Lobbying 

The Final Step 






A griLL-GNSOLVED PROBLEM 
CON(H^SSIONAL INVESTIGATIONS 



r 




(Continued) 

U. Following lecture^ hand out programs (CONCS^SS: FARi' l) to be started 
in class and completed at hov^. Check that programs are being taken 
to absent students. 

3* Announce that completed programs are due Monday. 

6. At the end at class period remind students to draw a line across the 
page of the program to show how much of the program they have completed. 

»«»«*«*«# 

Monday 

5 November DAY 2 LESSON PLAN FOR PERIODS 1, 2, Z, and 5 

1. List program topics (CONGRESS: PART l) on the blackboard. 

2. Collect completed programs (CONGRESS: PART l). 

3. Follow-up lecture/dlscusslon ( 20 to 30 minutes ) based on topics in 
programs completed as hcmework (CONG^SS: PART l): 

CONGRESS IS THE LAW-MAKING BRANCH OF OUR FEMRAL GOVERNMENT 

POWERS OF CONGRESS 

THE HOUSE OF REPRESENTATIVES 

Qualifications^ Term^ and Salary of Representatives 
Membership 

Reapportionment and Redistrlctlng 
Privileges of Congressmen 
Sole Powers of the House 
Strong and Weak Points of the House 

4. Hand out programs (CONGRESS: PART 2) to be started in class and completed 
at home. Check that programs are being taken to absent students. 

5* Hand out back programs to students who have been absent and failed to get 
programs at home. 

6. Announce that completed programs are due tomorrow. 

7. Check off students who handed in completed programs on Attendance and 
Assignment Control Sheet. Inform students who have not handed In 
programs that are due that they are delinquent. 

8. Review Lesson Critiques from CONGRESS: PART 1 to spot possible lecture 
topics not covered in class. 

9* At the end of class period remind students to draw a line across the page 
of the program to show how much of the program they have completed. 



Tuesday 

6 Wovember PERIODS 1, 2, 3» and 5 

1. List program topics (CX)NGRESS: PART 2) on the blackboard. 

2. Collect coopleted programs (CONGRESS: PART 2). 

3. Follow-up lecture/discussion ( 20 to 30 minutes) based on topics in 
programs completed as homework (CONGI^SS: PART 2) : 



THE SENATE 
Membership 

Qualifications, Term, and Salary of Senators 

Sole Powers of the Senate 

The Senate in Our Legislative System 

SESSIONS OF CONGRESS 

ORGANIZING CONGRESS 

Organization of the House i 

Committees of the House 
Party Organization in Congress 
Organization of the Senate 
Joint Standing Committees 
Legislative Reorganization Act 

4. Following lecture/discussion, hand out programs (CONGRESS: PART 3) to 
be started in class and completed at home. Check that programs are 
being taken to absent students. 




5. Hsind out back progrcuns to students who have been absent and failed to 
get programs at home. 

6. Announce that completed programs are due tomorrow. 

7. Check off students who handed in completed programs on Attendance and 
Assignment Control Sheet. Inform students who have not handed in 
programs that are due that they are delinquent. 

8. Review Lesson Critiques from CONGRESS: PART 2 to spot possible lecture 
topics not covered in class. 

9. At the end of class period remind students to draw a line across the 
page of the program to show how much of the progrcun they have completed. 
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Wednesday 
7 November 



DAY k LESSON PLAN FOR PERIODS X, 2, 3, 5 



1. Collect completed programs (CONGRESS: VABT 3) 

2. Films: Meet Your Federal Government ( l5 minutes) 

The Congress ( 20 minutes) 

3* Check off students who handed in completed programs on Attendance and 
Assignment Control Sheet. Inform students who have not handed in 
programs that are due that they cure delinquent. 

4. Review Lesson Critiques from CONGRESS: PART 3 to spot possible lecture 
topics not covered in class. 

5. Discuss yesterday's election. 

6. Hand out back programs to students who have been absent and failed to 
get programs at home. Announce that the completed back programs are 
due tomorrow. 

*«»«««** 

Thursday 

8 November DAY 5 LESSON PLAN FOR PERIODS 1, 2, 3, and 5 

1. List program topics (CONGRESS: PART 3) on the blackboard. 

2. Collect programs completed late and check off students who handed in 
programs on Attendance and Assignment Control Sheet. Inform students 
who have not handed in programs that are due that they are delinquent. 

3. Follow-up lecture/dlscusslon ( 20 to 30 minutes) based on topics in 
programs completed as homework (CONGRESS: PART 3): 

KEEPING A RECORD OF PROCEEDINGS 

HOW A BILL BECOMES A LAW 

Introducing a Bill 

Procedure in the House 

Procedure in the Senate 

The Conference Committee 

Log-Rolling 

Lobbying 

The Final Step 

A STILL-UNSOLVED PROBLEM 

CONGRESSIONAL INVESTIGATIONS 

4. Announce that there will be an exam tomorrow on Congress (topics covered 
by the three programs). 

5. Hand back three programs (CONGRESS: PARTS 1^ 2, and 3) for review. Check 
that programs are being taken to absent students. 
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(Continued) 

6. Hand out back programs to students vho have been absent and failed to get 
programs at home. 

7« Announce that all programs are to be handed back In tomorrow. 

8. Spend rest of period on a general recap of Congress ^ drawing on lesson 
critique notes If appropriate. 

««««#««« 

Friday 

9 November DAY 6 LESSON PLAN FOR PERIODS 1, 2, 3, and 5 



1. Collect three programs (CONCStESS: PARTS 1, 2, and 3) that had been handed 
back for review and any late programs. 

2. Hand out exam on Congress. 

3. Announce that when students conplete exams they should hand them In and 
pick up questionnaires. 

k. Check off students who handed back review programs and any late programs 
on Attendance and Assignment Control Sheets. Inform students idio have 
not handed in programs that are due that they are delinquent. 

5* About 5 minutes before the end of class collect the tests that haven't 
already been handed in. 



6. Hand out questionnaires to students who haven't already received them. 



?• Collect questionnaires at end of period. 

#««»«*«« 

Tuesday 

13 November DAY 7 LESSON PLAN FOR PERIODS 1, 2, 3, and 5 
1. Review exam on Congress. 



2. Introductory l^ictxire (about 15 minutes ) based on topics In four programs 
on the Presldeat and the National Administration: 

THE EXECUTIVE DEPARTMENT 



ELECTING THE PRESIDENT 

Nominating Methods 

Presidential -Preference Primaries 

The National Convention 

The Campaign 

The Election 

Election Disputes 
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DAY 7 (Continued) 



THE PRESIDENT 

Qualifications, Term, and Salary of the President 

Presidential Immunities 

Removal from Office 

The Presidential Succession 

THE EXECUTIVE OFFICE 

Administrative Assistants 
National Security Council 
Economic Advisers 

THE PRESIDENT AND HIS POWERS 

THE PRESIDENT'S EXECUTIVE POWERS 

Power of Appointment and Removal 
The President as Commander -in -Chief 
Conducting Foreign Relations 
Making Treaties and Executive Agreements 



THE PRESIDENT'S LEGISLATIVE POms 

Messages to Congress 
The Veto Power 

Patronage and National Leadership 
THE PRESIDENT'S JUDICIAL POWERS 
THE NATIONAL AOCNISTRATION 

THE CABINET OFFICERS AND THEIR EXECUTIVE DEPARTMENTS 

The Department of State 
The Treasury Department 
The Department of Defense 
The Department of Justice 
The Post Office Department 
The Department of the Interior 
The Depeu*tment of Agriculture 
The Department of Commerce 
The Department of Labor 

The Department of Health, Education, and Welfare 



INDEPENDENT AGENCIES 



3. 



The Civil Service Commission 
Other Independent Agencies 



REFORM MOVEMENTS 

Following lecture, hand out programs (THE PRESIDENT AND THE NAnONAL 
ADMINISTRATION: PART l) to be started in class and completed at home. 
Check that programs are being taken to absent students. 
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BAY 7 (Continued) 

SSmSSSSSMI w# 

U. Announce that completed programs are due tomorrow. 

5* At the end of class period remind sti^ents to draw a line across the 
page of the program to show how lauch'bf the program they have completed. 

****««*« 

WcdnG 8 doy 

ik November DAY 8 LESSON PLAN FOR PERIODS 1, 2, 3# and 5 

1. List program topics (THE PRESIDENT AND THE NATIONAL ADMINISTRATION: PART l) 
on the blackboard. 

2. Collect completed programs (THE PRESIDENT AND THE NATIONAL ADMINISTRATION: 
PART l). 

3* Follow-up lecture/discussion ( 20 to 30 minutes) based on topics in programs 
completed as homework (THE PRESIDENT AND THE NATIONAL ADMINISTRATION: PART l) 

THE EXECUTIVE DEPARTMENT 

ELECTING THE PRESIDENT 

Nominating Methods 

Presidential -Preference Primaries 

The National Convention 

The Campaign 

The Election 

Election Disputes 

THE PRESIDENT 

Qualifications^ Term^ and Salary of the President 

Presidential Immunities 

Removal from Office 

The Presidential Succession 

THE EXECtTTIVE OFFICE 

Administrative Assistants 
National Security Council 
Economic Advisers 

k. Hand out programs (THE PRESIDENT AND THE NATIONAL ADMINISTRATION: PART 2) 
to be started in class and completed at home. Check that programs are 
being taken to absent students. 

5. Hand out back programs to students who have been absent and failed to get 
programs at home. 

6. Announce that completed programs are due tomorrow. 

7. Check off students who handed in completed programs on Attendance and 
Assignment Control Sheet. Inform students who have not handed in 
programs that are due that they are delinquent. 



DAIJB (continued) ^ 

8 . Review Lesson Critiques from THE PRESIDENT AND THE HATIONAL AIMENISTRATION : 
PART 1 to spot possible lecture topics not covered in class. 

9 , At the end of class period remind students to draw a line across the page 
of the program to show how much of the progrsa they have completed. 

*#****♦♦ 

Thursday 

l^Now^w DAY 9 LESSON PLAN FOR PERIODS 1> 2, 3> 5 

1. List program topics (THE PRESIDENT AND THE NATIONAL ADMINISTRATION: 

PART 2 ) on the blackboard. 

2. Collect completed programs (THE PRESIDENT AND THE NATIONAL ADMINISTRATION: 
PART 2). 

3 . Follow-up lecture/discussion ( 20 to 30 minutes ) based on topics in programs 
completed as homework (THE PRESIDENT AND THE NATIONAL ADMINISTRATION: 

PART 2) : 

THE PRESIDENT AND HIS POWERS 

THE PRESIDENT’S EXECUTIVE POWERS 

Power of Appointment and Removal 
The President as Commander-In-Chief 
Conducting Foreign Relations 
Making Treaties and Executive Agreements 

THE PRESIDENT'S LEGISLATIVE POWERS 

Messages to Congress 
The Veto Power 

Patronage and National Leadership 
THE PRESIDENT'S JUDICIAL POWERS 

U. Following lecture/discussion, hand out programs (THE PRESIDEN T AN D THE 
NATIONAL ADMINISTRATION: PART 3 ) to be started in class and coiapleted 
at home. Check that programs are being taken to absent students. 

5 . Hand out back programs to students who have been absent and failed to 
get programs at home. 

6. Announce that conqpleted programs are due tomorrow. 

7* Check off students who handed in con 5 >leted programs on Attendance and 
Assignment Control Sheet. Inform students who have not handed in 
programs that ere due that they core delinquent. 

8 . Review Lesson Critiques from THE PRESIDENT AND THE NATIONAL ADMINISTRATION: 
PART 2 to spot possible lecture topics not covered in class. 
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DAY 9 (Continued) 

9 . At the end oY class period remind students to draw a line across the 
page of the program to show how much of the program they have completed. 

#««««««» 

Friday 

16 November DAY 10 LESSON PLAN FOR PERIODS 1, 2 , 3 , and 5 



1. List program topics (THE PRESIDENT AND THE NATIONAL AIMNISTRATION: PART 3 ) 
on the blackboard. 

2. Collect completed programs (THE PRESIDENT AND THE NATIONAL AIMINISTRATION: 
PART 3) . 

* <• ' 

3 . Follow-up lecture/dlscusslon ( 20 to 30 minutes) based on topics in pi* *ograni« 
completed as homework (THE PRESIDENT AND THE NATIONAL ADMINISTRATION: 

PART 3 ) : 

THE NATIONAL AIMINISTRATION 

THE CABINET OFFICERS AND THEIR EXECUTIVE DEPARTMENTS 

The Department of State 
The Treasury Department 
The Department of Defense 
The Department of Justice 
The Post Office Department 

4. Hand out programs (THE PRESIDENT AND THE NATIONAL ADMINISTRAnON : PART 4) 
to be started in class and completed at home. Check that programs are 
being taken to absent students. 

5 . Hand out back programs to students who have been absent and failed to get 
programs at home. 

6. Hand back three programs (THE PRESIDENT AND THE NATIONAL AIMINISTRATION: 
PARTS 1, 2, and 3 ) ^or review. Check that programs are being taken to 
absent students* 

7 . Check off students who handed in completed programs on Attendance and 
Assignment Control Sheet. Inform students who have not handed In programs 
that are due that they are delinquent. 

8. Review Lesson Critiques from THE PRESIDENT AND THE NATIONAL AMNISIRATION: 
PART 3 to spot possible lecture topics not covered in class. 

9 . At the end of class period remind students to draw a line across the page 
of the program to show how much of the program they have completed. 






Mond&y 

19 November DAY 11 LESSON PLAN FOR PERIODS 1, 2, 3, and 5 



1. List program topics (THE PRESIDENT AND THE NATIONAL ADMINISTRATION: 

PART 4) on the blackboard. 

2. Check off students who completed programs (THE PRESIDENT AND THE NATIONAL 
ADMINISTRATION: PART 4) on Attendance and Assignment Control Sheet. 

Inform students who have not completed programs that are due that they 
are delinquent. Tear off Lesson Critiques from the programs for use in 
lecture/discussion , 

3. Collect programs completed late and check off students who handed in 
programs on Attendance and Assignment Control Sheet. Inform students 

who have not handed in back programs that are due that they are delinquent . 

4. Follow-up lecture /discussion ( 20 to 30 minute^) based on topics in programs 
completed as homework (THE PRESIDENT AND THE NATIONAL ADMINISTRATION: 

PART 4) ! 

The Department of the Interior 
The Itepartment of Agriculture 
The Department of Commerce 
Thil Department of Labor 

The Department of Health, Education, and Welfare 

INDEPENDENT AGENCIES 

The Civil Service Commission 
Other Independent Agencies 

REFORM MOVEMENTS 

5* Announce that there will be an exam tomorrow on The President and the 
National Administration (topics covered by the four programs) . 

6. Hand back three programs (THE PRESIDENT AND THE NATIONAL ADMINISTRATION: 
PARTS 1, 2, and 3) for review. Check that programs are being taken to 
absent students. 

7. Hand out back programs to students who have been absent and failed to get 
programs at home. 

8. Announce that all programs are to be handed back in tomorrow. 

9. Spend rest of period on a general recap of The President and the National 
Administration, drawing on Lesson Critique notes if appropriate. 

******** 

Tuesday 

20 November DAY 12 LESSON PLAN FOR PERIODS 1, 2, 3, and 5 

1. Collect four programs (THE PRESIDENT AND THE NATIONAL ADMINISTRATION: 

PARTS 1, 2, 3, and 4) that had been handed back for review, and any late 
programs. 
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DWja (Continued) 

2. Hand out exam on the President and the National Administration. 

3. Announce that when students complete exams they should hand them In 
and pick \xp questionnaires. 

U. Check off students who handed hack review programs and any late programs 
on Attendance and Assignment Sheets. Inform students who have not 
handed in programs that are due that they are delinquent. 

5s About 5 minutes before the end of class collect the tests that haven *t 
already been handed in. 

6. Hand out questionnaires to students who haven't already received them. 

7. Collect questionnaires at end of period. 

Wednesday 

21 November DAY 13 LESSON PLAN FOR PERIODS 1, 2, 3, and 5 

1. Review exam on the President and the National Administration. 

2. Film; The Vice-President, Part 1 ( 30 minutes) . 

Monday 

26 November DAY ik LESSON PLAN FOR PERIODS 1, 2, 3, and 5 

1. Film: The Vice-President, Part 2 ( 30 minutes) . 

2. If time permits, begin introductory lecture ( about 15 minutes ) on topics 
in the program on Our Federal Court System: 

OUR FEDERAL COURT SYSTEM 

WAYS OF CLASSIFYING FEDERAL COURTS 
Levels of Courts 

Constitutional and Legislative Courts 
Courts of Original and Appellate Jurisdiction 

THE SUPREME COURT 

THE three chief LEGISLATIVE COURTS 

U.S. Customs Court 

Court of Customs and Patent Appeals 

Tax Court of the United States 

OTHER POWERS OP THE FEDERAL COURTS 



FEDERAL COURT OFFICIALS 



Tuesday 

27 Novenber DAY 15 LESSON PLAN FOR PERIODS 1, 2, 3, and 5 

1. If necessary^ complete Introductory lecture on Our Federal Court System. 

2. Following lecture^ hand out programs (OUR FEDERAL COURT SYSTEM) to be 
started In class and completed at home. Check that programs are being 
taken to absent students. 

3. Announce that completed programs due tomorrow. 

4. At the end of class period remind students to draw a line across the 
page of the program to show how much of the program they have completed. 

*#»««««« 

Wednesday 

28 November DAY l6 LESSON PLAN FOR PERIODS 1, 2, 3> and 5 

1. List program topics (OUR FEDERAL COURT SYSTEM) on the blackboard. 

2. Check off students who completed programs (OUR FEDERAL COURT SYSTEM) on 
Attendance and Assignment Control Sheet. Inform students who haye..uQt 
completed programs that are due that they are delinquent. Tear off 
Lesson Critiques from the programs for use In lecture/discussion. 

3. Follow-up lecture/dlscusslon ( 20 to 30 minutes ) based on topics in 
programs cooqjleted as homework (OUR FEDERAL COURT SYSTEM). 

OUR FEDERAL COURT SYSTEM 

WAYS OF CLASSIFYING FEDERAL COURTS 
Levels of Courts 

Constitutional and Legislative Courts 
Courts of Original and Appellate Jurisdiction 

THE SUPREME COURT 

THE THREE CHIEF LEGISLATIVE COURTS 

U.S. Customs Coiu*t 

Court of Customs and Patent Appeals 

Tax Court of the United States 

OTHER POWERS OF THE FEDERAL COURTS 

FEDERAL COURT OFFICIALS 

4. Announce that there will be an exam tomorrow on Our Federal Court System 
(topics covered by the program). 

5. Hand out back programs to students who have been absent and failed to get 
programs at home. 

6. Announce that all programs are to be handed In tomorrow. 

7. Spend rest of period on a general recap of Our Federal Court System^ drawing 
on Lesson Critiques If appropriate. 



Thursday 

29 November DAY 17 LESSON PLAN FOR PERIODS 1, 2, 3, and 5 

1. Collect programs (OUR FEDERAL COURT SYSTEM) that had been used for review, 
and any late programs. 

2. Hand out exam on Our Federal Court System. 

3. Announce that when students complete exams they should hand them in and 
pick up questionnaires. 

U. Check off students who handed back review program^ and any late programs 
on Attendance and Assignment Control Sheets. Inform students who have not 
handed in programs that are due that they are {delinquent. 

*1 

5. About 5 minutes before the end of class collect the tests that haven *t 
already been handed in. 

6. Hand out questionnaires to students who haven't already received them. 

7. Collect questionnaires at end of period. 



AFFEliDIX £ 



ATOmtANCE AliD ASSIGNMERT COKTROL SHEET 

(The following form was used by teachers to accurately monitor the completion 
of programs.) 



AaaENMNCE Ain) ASSIGKMEINT CCMROL 3 HEET 



Monday Tuesday Wednesday Thursday Friday 

( ) ( ) ( ) ( ) ( ) 

(Dates noted, in parentheses under each day of the school week.) 

(Nature of assignioent noted herej eg.> CONGRESS; £ART 1^ etc.) 



STUDENT m/SSS 



IN OUT IN OUT IN OUT IN COT IN OUT 



APPENDIX P 



SAMPLES OF WRI 1 E«IN PROGRAMS 

(The foUo¥ing are sanple pages from, the write-in programs used in the 
Fall 1962 field test. Three program booklets totaling 55 pages were used 
for the segment on Congress; four program booklets totaling 60 pages were 
used for the segment on The President and the National Administration; 

one pro^am booklet of I7 pages was used for the segment on Our Federal 
Court System .) — — . 



Ni 
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THE COHFERENCE CX»«MITTBE 

A bill passed by one house does ndt always pass the other house in 
the same form in which it passed the first. Sometimes a bill passes 

one in a slightly different form from the form in 

which it has p the other. But to become a law, a 

bill must pass both houses in the same f So when there 

are between the final Senate and 

versions of a bill, it must go to a Conference Committee. The 
presiding officer of each house appoints members from his house to 
sit on a committee to iron out these differences. This committee is 
called the C C 

The purpose of the Conference Committee is to eliminate or get rid 

of between the Senate and the 

versions of a bill that has already ^o"th houses. 

If a bill has passed both houses in the same form there is no need of 
a . A Conference 

Committee is called a Conference Committee because it is a committee 
that c on how to make the Senate and House versions 

of a the same. 



After the Conference Committee has agreed on a version of the bill that 
it thinks will be acceptable to both < then the two 

houses have to vote again on the bill. Usually they both 
to pass it. 

Neither house can amend or change the version of a bill that has been 
worked out by the Conference Committee. It must be either 

or rejected in the form that the Conference 
Committee has decided on, though they can send it back to the 
C c and ask them to improve it. 

About one bill in ten passes the House and in 

different versions and therefore goes to a _■ . 

• Since the version suggested by this committee 

^ be changed, this gives a good deal of power to the Conference 
(can not ^ ' 

Committee . 




9 



A BEVIEW 



• The ex branch of our Federal government is now a 

large and very complex organization. Immediately under the President 
'* is his Cabinet. Each of these C officers is also the 

head of a major executive d There are ten regular 

C members > which means that there also are ( number) 

departments. Cabinet officers are appointed by 
, but the upper house of Congress^ the 
. must approve them. 

Apwrt from these ten executive departments. Congress has created various 

independent b and c take care 

of other types of government activity. One of these i 

commissions is the Civil Service Commission, which has the responsibility 

of finding the most competent people to fill g . 

positions. Most government jobs are filled by people chosen by the 
' Commission. 



There are a few helpers which the President is able to appoint without 

the approval of the S . These people are his administrative 

assistants. A ^lelp 

the by collecting information for him. 



The P gets ec financial 

advice from the Bureau of the Budget and the Council of Economic Advisers. 
How much money the government takes in and pays out is the concern of the 

B whereas the Council 

of is more concerned with 

matters which affect the economic condition of the entire country. 

Finally, if the President wanted advice on matters concerning national 
defense, he would take this up with the National Council 

which is concerned with our nation *s security. 

Many refortas have been made in the ex branch of our 

Federal government, but still more are being considered. 
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case is carried higher to the • 

If eight Judges try the case and four of them think the verdict 
of the lower court was right and four think it was wrong, th6n 
^the man is acquitted, 

(the decision of the lower court stands, and he is not acquitted. 
So if half the Supreme Court Judges decide one thing and the other 

half decide the opposite thing, it is called a vote 

and the decision of the lower court is upheld. But if one man 
^ore than half of all the Supreme Court Judges trying the case come 

to a certain decision, it is called a si_ vote and 

whatever this majority vote decides "hecomes the S 
[b decision. 



Justices who disagree with the majority opinion sometimes write a 

''dissenting opinion" explaining why they think the decision was 
(right. A 4 

(wrong. ^ Justice who writes a dissenting opinion on a case is 
not in the but in the minority, A 



opinion is a m 



opinion. 



Sometimes all the Judges vote the same. When this happens, we say 

that the decision is unanimous. A 9 - 0 decision is a 

. In their unanimous decision on school 

desegregation, the nine Supreme Court voted 

( number) to (number) . 



The Supreme Court has both original and appellate Jurisdiction. 
The original Jurisdiction of the Supreme Court includes all cases 
regarding ministers, ambassadors, and consuls. These high 
officers of the Foreign Service are tried first in the 

. as a court of 



When the Supreme Court is the first court to try a case, it acts as 

a court of When it 

tries a case that has already been tried in a lower court, it acts 
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APPENDIX G 



LESSON CRITIQUE 

(The following is the form conpleted by students in PRCKSRAM classes before 
each program booklet was banded in. ) 




LESSON CRITIQUE 



NAME PERIOD DATE 

Write your reactions to the material covered in the program, under the 
headings below. Be as specific as possible. 

WHAT TOPICS ARE NOT CLEAR, OR HEED MORE EXPLANATION? 



WHAT TOPICS WOULD YOU LIKE TO KNOW MORE ABOUT? 



AFPE19DIX H 



CRITERION TESTS 

(The following are the criterion tests administered to FROQRAM and NON-'PROGSIAN 
classes during the Fall 1962 field test.) 






HAME 



PERIOD 



U«S. GOVERNMENT EXAMINATION : CONGRESS 



; 




4 



MULTIPLE CHOICE 



Please write the letter In 
front C3f the BEST answer in 
the blank at the left of 
each question. 



JL. Coflopared to Representatives^ Senators! 

A. can be younger 

B. have the same base pay 

C. have fewer fringe benefits 

D. serve a shorter term 

£• are less experienced 



2 . 



% 



A certain city has a rule saying that children must attend whatever 
school Is nearest their homes. The city builds new schools so 
that the colored children will all be neEurest to one particular 
school. This city is maintaining segregation by means of: 

A. filibustering 

B. gerrymandering 

C. closure 

B. pork-barrel legislation 



3* When population moved westward in the nineteen*ch century it 
affected the size of the Senate only because: 

A. some states lost population 

B. some states gained population 

C. new states entered the Union 

D. the western states had larger areas 



k. A U.S. Senator is elected to office for a term of: 

A. 2 years 

B. 4 years 

C. 6 years 

D. 4 sessions of Congress 



3* The Speaker of the House of Representatives is: 

A. appointed by the President 

B. elected by the House Rules Committee 

C. elected directly by the people 

D. chosen by the majority party In the house 



o 
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6 . 



Suppose a state lost population heavily but could not agree on 
redistricting. What would happen? 

A. it could not send any Representatives to Congress 
until it did redistrict 

B. an arbitrary redistricting would be done by 
Federal officials 

C. the old districting could be used> but a certain 
number of the elected Representatives would be 
elected from the state at large 

D. all representatives would be elected from the 
state at large 



7 . 



The actual drawing or setting of Congressional district boundarlee 
Is done by: 

A. the State Legislature 

B. the Census Bureau 

C. the Congress 

D. the President 



8. “Pro ten 5 >ore” means 

A. professional 

B. timely 

C. temporary 

D. appointed 



9 . 



A certain 
from each 
represent 

A. 

B. 

C. 

D. 

E. 



savage tribe is governed by a Council of Elders (one 
village) and by a group of warriors (each one chosen to 
100 natives). Its government is: 
bicameral 
tyrannous 
federal 
confederate 
monarchic 



10 0 A friend once said to a local candidate who was running for office, 
"You can*t miss . . . Eventually, that is. If you don’t make 
it this time, we’ll keep moving those boundaries until you do." 

He was suggesting: 

A. pork-barrel legislation 

B. log-rollikig 

C. a lockout 

B. a gerrymander 













11. Which of the following is a party organization, made up of 
members of only one party? 

A. Steering Conmlttee 

B. Rules Committee 

C. Legislative Committee 

D. Joint Standing Committee 



12. Impeachment proceedings can he begun only by: 

A. the Senate 

B. the House 

C. two -thirds of each house, voting separately 

D. two -thirds of Loth houses, voting together 

E . the President 



13. 



When we say someone is Impeached, we mean he is: 

A. accused 

B. tried 

C. convicted 

D. punished 



% 



l4. Tte "upper house" of our national legislature is called: 

A. Congress 

B. the Senate 

C. the House of Repre&entatives 

D. the National Assombly 



15. The means designed to provide adequate representation for everyone 
in spite of population shifts is called: 

A. gerrymandering 

B. log-rolling 

C. reapportionment 

D. patronage 



16. Congressman Blanh sent all his Christmas cards without paying any 
postage on them. In doing this he made wrong use of: 

A. Congressional immunity 

B. the franking privilege 

C. seniority rule 

D. lobbying 



17 • A Congressman could be punished for: 

A. a speech In the Senate saying that television 
manufacturers are over -charging the public 

B. an attack In committee on the John Birch Society 

C. condemning an Individual by name In a speech 
given In Congress 

D. a traffic violation 



l8. Which one of the following Is sometimes considered as a weak 
point of the House of Representatives rather than of the 
Senate? 

A, small size 

B. short term of office 

C, failure to represent city areas 

D. Inability to Introduce money bills 



19. 



Which one of the following Is a sole power of the House of 
Representatives 7 

A. to elect the Vice-President when the Electoral College 
cannot . 

B. to Impeach a Federal judge who Is said to have taken 
bribes 

C. to vote on appointments made by the President 

D. to ratify treaties 



20 . 



With the passage of time the number of people represented by a 
member of the House of Representatives has: 

A. decreased 

B. Increased 

C. remained the same 

D. fluctuated 



21 . 



The Congress of the United States: 

A. Is the law-making branch of our Federal 
Government 

B. Is unilateral In structure 

C. Is provided for In Article III of the 
Constitution 

D. Is responsible for administering and 
Interpreting the laws 



III 



m 






22 . 



r^ 









The House and the Senate have many standing comml.ttees to discuss 
proposed legislation. Each standing committee deals with a 
different : 



A. part of the world 

B. part of the country 

C. version of the proposed bill 

D. kind of subject 

E. aspect of the proposed bill 



23. Which one of the following Congressmen Is most likely to be made 
chairman of the Committee on Agriculture In the House of 
Representatives 7 

A. one from the minority party j 

B. one who has a farm of his own j 

C. one who has been a member of the House for 
many years 

D. one who has a close acquaintance with the 
President 



2 k, If California's population grows to four times its present size 
by the year 2000, but the country in general has only twice as 
much population, about how many representatives will California 
have in the year 2000? 

A. twice as many 

B. only a few more than now 

C. the same number as now 

D. fewer than now 



25. A man is elected Senator but Is refused entrance to the Senate 
because of his membership In a certain society. Who refuses 
him entrance? 

A. the President 

B. the Senate 

C. Congress 

D. the Supreme Court 



26. A bill to Increase Immigration: 

A. must originate In the House 

B. must originate In the Senate 

C. may originate In either house 

D. would be unconstitutional 






















Atten^ts have been made to regulate lobbying by: 

A. the closure rule 

B. requiring all lobbyists to register 
C« making professional lobbying illegal 

B* printing all statements made by lobbyists 






28 . 



To become a lav, a bill must always be approved by: 

A. one house of Congress and the President 

B. both houses of Congress and the President 

C. both houses of Congress 

D. the President 



29 . 



Members of the House may use all of these means of voting except ; 

A. viva voce 

B. rising 

C. Australian ballot 

D. roll call 



30 . 



What happens If a bill passes the House of Representatives in 
one form and the Senate in another fonn? 

A. the President must decide which to sign and 
which to veto 

B. the differences are eliminated by the Conference 
Committee 

C. the Senate and House vote together as to which 
form to adopt 

B. the Senate version is adopted 

E. the House version Is adopted 



31 . 



The number of members required in attendance before official 
business can be transacted Is called a: 

A. committee of the whole 

B. plurality 

C. caucus 
B. quorum 

E. majority 
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32. The President can use a pocket veto: 

A. by holding a bill for ten days 

B. when Congress adjourns within ten days 
after passing a bill 

C. to block any legislation he does not 
approve of 

D. by a veto within ten days after a bill 
Is passed by Congress 



33. The Rules Committee Is the most Important political body In the 
House because: 

A. It controls the time allotted to debate 
on bills 

B. It can prevent a bill from being considered 
In the House 

C. It can change the power and Jurisdiction of 
the various House committees whenever It wishes 

B. It determines the order In which bills may be 
considered 

E. all of the reasons given above are true 




Division of labor, so that specialization Is possible, Is the 
reason for our legislature's: 

A. bicameral system 

B. bipartisan system 

C. party system 

iO. committee system 

E. system of checks and balances 




The graveyard of proposed laws Is the: 

A. House of Representatives 

B. Senate 

C . White House 

D. standing committee 



36. As a Congressman, If you wished to have a bill written, you would 
consult : 

A. the Legislative Reference Service 

B« the Speaker 

C. the Private Calendar 

D. an attorney 



37. Senator Blank from a uranlum-mlnlng Rocky Mountain state agrees 
to vote for Federally -financed apartments in the great cities 
if Senator So-and-so from New York will vote for Federal support 
of Turanium prices. The two senators are engaging In: 

A. boondoggling 

B. junketing 

C. log-rolling 

D. lobbying 

E . filibustering 



38 . 



Which one 
Senate? 

A. 

B. 

C. 

D. 

E. 



of the following bills could not be introduced in the 

one to send an American ballet troupe abroad 
one to raise the tax on women's handbags 
one to deepen an Alaska harbor 
one to lower the voting age 
one to fix railroad rates 



39« A filibuster is used: 

A. only in the Senate 

B. in the Senate and House 

C. only in the House 

D. in neither the Senate nor the House 



o 

ERIC 



Lls't powers given to Congress in Article I ^ Section 8 ot the Constitution 



1 . 



2 . 



3 . 



k. 



5 . 



6 . 



7 . 



8 . 



9 . 
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TRUE - FALSE Please write T or F In the blank 

at the left of each statement. If 
a statement is partially false con- 
sider it totally false. 



1. The floor leader manages his party in Congress^ plans strategy^ 
and allots time to the party's speakers. 

2. The President decides the date on which Congress is to adjourn when 
the two houses cannot agree on it. 

3* The President of the Senate is Lyndon Johnson. 

4. Congress is empowered to make Congressional Investigations Into 
any sphere of our lives. 

3 . The House of Representatives has a higher status and more prestige 
than the Senate. 



6. The periodic enumeration or count of the people is called a census. 



7 . Very special training is necessary in order to qualify for election 
as a member of the legislative branch of om* government. 

Q, Both houses of Congress must pass on all appointments made by the 
President. 



9 . By a two-thirds majority, the Senate must consent to treaties made 
by the President before they can go into effect. 

10. One method of nominating members of Congress is known as direct 
primary. 



11. The purpose of Congressional immunity from libel suits Is to make 
sure that Congressmen have freedom of speech. 

12. The party whip polls party menibers to determine the support of bills. 



13 . A bill that will involve the spending of money must originate in 
the Senate . 

1 4 . The Senate and the House have to vote on a bill in the form it is 
received from the Conference Committee. 



15. If the President vetoes a bill, it requires a three -fourths vote 
of both houses to over-ride his veto. 



16. The Closure Rule was adopted to restrict the activities of lobbyists. 



,17. Unlimited debate is one of the ways that a minority group can 
maintain an influence in government. 



18. Bills concerning individuals are placed on a list called the 
Private Calendar. 

19. Pork-barrel legislation refers to bills which are referred to 
conference committees 

20 . When an entire legislative body meets as a committee it is called 
a committee of the whole. 

21 . A bill, to become law, has to pass the lower house first and then 
the upper house. 




PERIOD 



U.S, GO\SRHMENT EXAMINATION: THE PRESIDENT AND THE NATIONAL AMNISTRATION 



NAME 




t 













MULTIPLE CHOICE 



Please write the letter in 
front of the BEST answer 
in the blank at the left 
of each question. 



1 . 



To win the election a candidate for the Presidency must have 
at least; 



A. a majority of the electoral votes 

B. two-thirds of the electoral votes 

C. a majority of the popular votes 

D. a plurality of the electoral votes 



2. In most states the delegates to the national conventions that 
choose the presidential candidates are chosen by: 

A. the people 

B. the state party conventions 

C. the previous national convention 

D. the County Board of Supervisors 



3. A bill passes Congress and goes to the President for approval 
or veto. Suppose the President is generally in favor of the 
bill but thinks it takes in too much territory. Can he approve 
it in modified form? 

A. Yes, if he has already vetoed it once 

B. Yes, if Congress does not pass it again in 
its original form 

C. Yes, if it would cost no more in the new form 

D. No 



k. According to the Hatch Act; 

A. no federal employee may participate in a political 
campaign 

B. no one may give over $5000 to any political 
committee 

C. persons on relief are not allowed to vote 

D. time off from the job must be given for voting 

E. campaign workers cannot be paid 



5 . If the President appointed a new head for the Central Intelligence 
Agency, the appointment would have to be approved by; 

A. the House of Representatives 

B . the Senate 

C. both Houses of Congress 

D. the Electoral College 

E. the Central Intelligence Agency 
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6. The Speaker of the House is next in line after the Vice-President, 
to succeed to the Presidency after the President's death. 

Formerly the Secretary of State was next in line. The change 
was made because: 

A. it is essential for the person succeeding to the 
Presidency to be of the same party as the 
President in order to provide continuity 

B. there is always a Speaker of the House but there 
is not always a Secretary of State 

C. it was felt that the Secretary of State already 
had too much power 

D. the Secretary of State is named by appointment 
and therefore may not reflect the will of the 
people 

7. In the Electoral College each state of the Union has at least 



how many votes? 


A. 


One 


B. 


Two 


C. 


Three 


D. 


Four 



8. The State of Colorado sends eight representatives to the House 
of Representatives. How many votes does it have in the 
Electoral College? 

A. One 

B. Three 

C. Eight 

D. Ten 

E. Not enough information is given to answer this 
question 



9. 



The only President to serve more than two terms (full terms) was 

A. Woodrow Wilson 

B. Franklin D. Roosevelt 

C. Harry Truman 

D. Theodore Roosevelt 



10. If the Cabinet agrees unanimously on a certain course of action, 
the President: 

A. must follow it 

B. must follow it if he cannot convince at least 
1/3 of the Cabinet that his point of view is right 

C. can do as he likes regardless of what the Cabinet 
thinks 



•• 



11, The Bureau ocf Mines would be concerned with all of the following 
except ; 

A. raising miners' wages 

B. preventing waste of natural gas 

C. production of a rare gas 

D. development of improved safety helmets 
for miners 

E* checking the quality of coal used In 
the furnaces of Army barracks 



12. If the Secretary of State Is to be fired, who fires him? 

A. The Department of State 

B. The President 

C. The Senate 

D. The President and the Senate 

E. Both houses of Congress voting together 



13. The Independent offices, agencies, and commissions are called 
independent because they are Independent of : 

A. the President 

B. Congress 

C. the major government departments 

D. public opinion 

E . state rule 



purpose of the Department of Agriculture Is to: 
control agricultural surpluses 
Increase the efficiency of agricult\n:al 
production 

control harmful Insects and plant diseases 

carry out an educational program 

set prices for food and fiber products 



15. All of the following €ure independent agencies except the: 

A. Federal Civil Service Commission 

B. Interstate Commerce Commission 

C. Federal Communications Commission 

D. National Labor Relations Board 

E. Bureau of Mines 



l4. The chief 

A. 

B. 

C. 

D. 

E. 



16. The DepetPtment of Labor is most concerned with: 

A. solving unemployment problems arising from 
automation 

B. keeping wages low enough for American 
manufacturers to compete with those of Europe 

C. protecting laborers* voting rights 

D. relaxing the immigration lavs 

E. providing pensions for disabled veterans 



17. If it were not that they already have too much work to do without 
it, the work of the various independent hoards and. commissions of 
the United States government would be done by the: 

A. Senators 

B. Representatives 

C. Supreme Court judges 

D. governors of the various states 

E . executive departments 



l8. The legal arm of our Federal Government is: 

A. the F.B.I. 

B. the Department of Justice 

C. the Supreme Court 

D. the legislative branch of the government 



19. The Narcotics Bureau is one of the divisions of: 

A. the Department of Commerce 

B. the Department of Labor 

C. the Department of Health, Education, and Welfare 
D* the Treasury Department 



20. If you were Secretary of State today, which of the following 
would, not be a responsibility of your department? 

A. Publish all the laws passed by Congress 

B. Publish all the proclamations issued by 
the President 

C. Administer the Foreign Service 

D. Supervise the collection of tariffs on goods 



21. Social security is administered, by the Department of: 
A* the Interior 

B. Labor 

C. the Treasury 

D. Health, Education, and Welfare 

E . Commerce 



22. If the United States government's State Department follows a 
foreign policy toward Latin America that is a failure, who 
ultimately is responsible? 

A. The Cabinet 

B. The Secretary of State 

C. The President 

D. Congress 

E. The Foreign Service 




The Treasury Department of the United States has the power to: 

A. lay taxes 

B. collect taxes 

C. Impose duties 

D. to borrow money on the credit of the United 
States 



2k, 



25 . 



Which one 


of the following is not an e: 


A. 


Attorney -General 


B. 


Secretary of the Interior 


C. 


Secretary of War 


D. 


Postmaster General 


The heads 


of executive departments are 


A. 


Foreign Service officers 


B. 


Cabinet members 


C. 


civil -service appointees 


D. 


legislators 


E. 


active party members 



26. The chairman of the Joint Chiefs of Staff is: 

A. a civilian appointed by the President 

B. a military officer 

C. head of the Departments of the Army, Navy, 
or Air Force 

D. a military man appointed by the Secretary 
of Defense 

E. the Secretary of Defense 



27. 



The President and Vice-President are guarded chiefly by the: 

A. FBI 

B. Civil Service 

C. Secret Service 

D. Coast Guard 

E . Marines 



28 . 



The Department of Defense was established to: 

A. improve co-operation among the armed forces 

B. make it easier to negotiate with organizations 
such as NATO 

C. make it possible for all military affairs to 
be administered from the Pentagon 

D. prevent too great a centralization of 
military power 

E. give the President more power as commander- 
in-chief 
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29 • If you were an employee of the Bureau of Mines, you would be 
under : 

A. the Department of Agriculture 

B. the Department of Defense 

C. the Department of Commerce 

D. the Department of the Interior 



30. Co-operation among the military services has been made especially 
necessary by the: 

A. need for missile and space research 

B. reduced authority of the Joint Chiefs 
of Staff 

C. tax burden involved in payments to veterans 

D. need for extending defense contracts to 
small business 

E. need for forces that could fight small 
"brushfire" wars 



31. The Post Office Department loses money every year, chiefly because; 

A. the building of new post-offices is carried on 
in times of depression as a form of work relief 

B. the Post Office has to pay part of the expenses 
of the railroads and highway system over which 
the mail is carried 

C. the Post Office also supports a banking system 

D. a great deal of mail is carried at less than cost 



32. The President need not take the advice of his cabinet if; 

A. the vote does not reach a majority 

B. one cabinet member abstains from voting 

C. other competent advice contradicts it 

D. he disagrees with it 

E. Congress is opposed to it 



.33. 



If you were in the Secret Service, you would be working for: 

A. the Department of Justice and the F.B.I. 

B. the Navy Department 

C. the Treasury Department 

D. the State Department 



Most of the 
from: 

A. 

B. 

C. 

D. 



money for expensive presidential campaigns comes 

state tax funds 
congressional appropriations 
contributions of $1000 or less 
a very few wealthy individual contributors 



The powers of the President have been broadened over the years 
by the action of strong individual Presidents and by; 

A. the Constitution 

B. the State Legislatures 

C. agreement between the two major parties 

B. Congress 



Which one of the following states would lose most voting power 
by having a presidential election thrown into the House of 
Representatives ? 

A. Alabama 

B. Montana 

C. Vermont 

D. California 

E. West Virginia 



The Hoover 

A. 

B. 

C. 

D. 

E. 



Commission’s purpose was to; 
improve government efficiency 
strengthen international relations 
reorganize the Republican Party 
assist the under -developed nations 
insure civil rights 



Immigration and naturalization affairs are taken care of by the 
Department of: 

A. Health, Education and Welfare 

B. Labor 

C. State 

D. Defense 

E. Justice 



How are delegates to the national nominating conventions chosen 
in most of the states? 

A. By presidential -preference primaries 

B. By popular vote of all voters 

C. By state party conventions 

D. By caucus of party leaders in the 
state government 



A certain presidential candidate, from out of the state, wins 
the presidential -preference primary in a very small state. 

Of what advantage is this to him? 

A. The staters delegates are pledged to support 
him at the national convention 

B. He is sure of at least three votes in the 
Electoral College 

C. He becomes a "favorite -son" candidate 

D. The state party organization guarantees 
him campaign expenses 



The government tries to limit campaign contributions by any one 
person, chiefly because: 

A. it wants all persons to participate in 
citizenship activities 

B. if one person gave a large sum he might 
expect favors in return 

C. presidential campaigns are already too 
long and too much of a drain on the 
energy of the candidate 

D. large contributions to one party would 

put the other at too much of a disadvantage 



Power over the United States' foreign affairs is divided between 
the President and the: 

A. Congress 

B . Supreme Court 

C. Department of Defense 

D. state governments 

E. United Nations 



Bureaus of the Department of Commerce do all of the following 
except: 

A. set up qualifications for pilots of 
private planes 

B. predict hurricanes 

C. promote overseas trade 

D. protect the rights of inventors 

E. protect factory workers 



How does Congress usually exercise control over members of the 
Cabinet? 

A. By impeaching them 

B. Through the President 

C. By its power of limiting funds 

D. By referendum 

E. By the spoils system 



45. A continuing problem of our national defense establishment is: 

A. lack of a seat on the Cabinet 

B. the Army, Navy, and Air Force carrying 
on separate but similar work 

C. concentration of all research in the Ar?v 

D. military ownership of most of the means 
of defense production 

E. civilian control of atomic energy 



46. The head of the Department of Justice is known as the: 

A. Chief Justice 
B* Attorney General 

C. Secretary of Justice 

D. Adjutant 

E . Secretary General 



Write the name of the department headed by each of the following cabinet 
officers : 



1. Robert McNamara 

2. Luther Hodges 

3. Stewart Udall 
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Dean Rusk 



Classify the President *s Powers 



In Column B on the right are the different kinds of 
powers exercised by the President. On the left in 
Column A are some specific powers. Place the letter 
of the Correct type of power listed in Column B beside 
the number for each item in Column A. 

A B 

He appoints cabinet members A. Executive power 

B. Legislative powc 

He grants pardons C. Judicial power 

He delivers the State of the Union message 

He sends special messages to Congress 

He delivers the Budget message 

He vetoes bills 

He appoints Federal fudges 

He is Commander -in -Chief of the armed forces 

He guarantees to every state a republican form of government 

He conducts foreign affairs 

He may pocket -veto a bill 

He grants reprieves 

He negotiates treaties 

He grants amnesties 

He may suspend the writ of habeas corpus in extreme national 
emergency 



TRUE - FALSE 



Please write T or P in the blank 
at the left of each statement. If 
a statement is partially false con- 
sider it totally false. 



1. Only the President has the power to declare war. 



2. Congress can override a "pocket veto" by a two-thirds vote 
of both houses. 



3 . The National Chairman of each party is generally selected by 
its presidential candidate. 



4. Although the President may negotiate treaties with foreign 
countries, all such treaties must be ratified by a two- 
thirds vote of the Senate. 



5 . The President can restrict the power of the legislative branch 
through use of the veto but he has no control over the Judicial 
branch. 



( b . The President can accept part of a bill and veto part of it 
due to his "item veto" power. 



7. A presidential candidate can receive a majority of the popular 
votes in the nation and yet receive a minority of the electoral 
votes . 



8. The Cabinet shares with the President the responsibility for 
Congressional action in a time of national emergency. 



9 . The Patent Office is part of the Department of the Interior. 



The FBI is a division of the Treasury Department. 



Cabinet members have the right to appear before Congressional 
Committees and present their views but may not vote on bills. 



The President may dismiss a cabinet officer without the approval 
of the Senate. 



The Pure Pood and Drug Administration is under the Department of 
Commerce. 



The Department of Commerce is to business what the Department of 
Labor is to workers. 



Independent bureaus and commissions^ as well as administrative 
assistants to the President, are under the jurisdiction of the 
Departments of the Cabinet. 



The U.S, Constitution makes no provision for the President's 
Cabinet . 



The Treasury Department of the United States has the power to 
"lay and collect taxes, duties, imposts and excises." 



The Vice-President now sits in with the President's Cabinet 
and is a member of the National Security Council. 



Independent boards and commissions are usually created by the 
President. 



NAME PERIOD 
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U.S. C50VERNMEHT E3CAMINATION: OUR FEDERAL COURT SYSTEM 




ERIC 



MULTIJrUE CHOICE 



Please write the letter in front of 
the BEST answer in the blank at the 
left of each question. 



1 . 



How many Justices now sit on the Supreme Court of the United 
States? 

A. 10 

B. 6 

C. 9 

D. 12 



2. Failure to obey a court order or an attack on the dignity of 
the court may be ruled: 

A. treason 

B. contempt of court 

C. sedition 

D. unconstitutional 



3 . In addition to rendering decisions in the Supreme Court, Supreme 
Court Judges : 

A supervise the work of the appellate courts 

B. sit as non -voting members of the Senate 

C. advise the Senate and House legislative committees 

D. act as circuit Judges in the state courts 



If. If it were found that a new kind of Federal court was needed, who 
would establish it? 

A . The President 

B. Congress 

C. The Judges of the Supreme Court 

D. The people, through an amendment to the 

Constitution 



5* Which one of the following is an injunction? 

A. A court order to stop hauling dirt over 
residential streets 

B. A statement as to why a person has been 
imprisoned in the county Jail 

C. A merger between two companies that make the 
same type of product 

D. An order to pay a fine for contempt of court 



6. Federal District Courts vere established by: 

A. the Judiciary Act of 1789 

B. the Congress under the Articles of Confederation 

C, the First Ten Amendments 

D, the Supreme Court 



7. People who break the laws of the United States are usually 
brought before: 

A. Justices of the peace 

B. county courts 

C . state courts 

D. Federal District Courts 

E. appellate courts 



8, Federal courts ere classified as constitutional or legislative 
according to: 

A. the authority by which they were established 

B. the kind of cases they try 

C. the way their Judges are selected 

D. whether they have original or appellate Jurisdiction 

E. whether or not they can issue injunctions 



9* Which of the following is a Constitutional Court? 

A. Customs Court 

B . Consular Court 

C. Tax Court of the U.S. 

D. Supreme Court 



10. What is the main difference in function between state courts and 
Federal courts? 

A. Federal courts deal with violations of Federal law 
and state courts deal with violations of state law 

B, Federal courts deal with more important cases than 
state courts 

C. Cases can be appealed from a state but not from a 
Federal court 

D, Federal courts deal only with violations of the 
Constitution itself 
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11 . 



Which of the following are cases over which the Federal District 
Courts have no Jurisdiction? 

A. Cases involving ministers and ambassadors 

B. Cases involving internal revenue 

C. Cases in bankruptcy 

D. Anti -trust cases 



12 . 



Precedents established by Judges when they interpret and apply 
laws that have been passed by legislative bodies are called? 

A. criminal law 

B. civil law 

C. statutory law 

D. legislative law 

E. common law 




A court that 

A. 

B. 

C. 

D. 

E. 



has appellate Jurisdiction Is one that can: 
try a case for the first time 
decide on whether a case has been properly tried 
try Federal cases 
try non-Federal cases 
enforce its decisions 



lU. 



Who appeals 

A. 

B. 

C. 

D. 



a case from a District Court? 
The Judge 
The losing party 
The District Attorney 
The United States marshal 



15 . 



Courts of original Jurisdiction differ from courts of appellate 
Jurisdiction in that courts of original Jurisdiction: 

A. try offenses against both state and 
Federal laws 

B. try o:^*fenses only against state laws 

C. do not try cases where large amounts of 
money tre Involved 

D. try cases that have not been decided 
before 

E. try casee that have already been decided 
once or tore 






l6. Which of the following do not fall within the original 
jurisdiction of the Supreme Court? 

A. Suits brought against any one of the states 

B. Suits brought by a state against citizens of 
another state 

C. Cases involving violation of a Federal law 

D. Cases affecting ambassadors and ministers 



17* Which of the following has original jurisdiction? 

A. Court of Appeals 

B. District Court 

C. Court of Customs Appeal 

D. The General Court 



l8. "The Federal judiciary is organized much like a three-level 
pyramid." In this pyramid the middle level is the; 

A. United States Court of Appeals 

B. District Court of the United States 

C. United States Court of Claims 

D. Territorial Court 



19 . The U.S. Court of Claims deals with; 

A. criminal offenses against the Federal government 

B. political offenses against the government 

C. money claims against the government 

D. the government's claims against private individuals 



20 . 



In a Federal Court of Appeals a case is tried by a: 

A. jury 

B. single judge 

C. panel of judges 

D. board of legal experts in the field of 
government affected 

E . mediation board 



21 . 



All of the following are Federal courts except the: 

A. District Court 

B. Juvenile Court 

C. Court of Claims 

D. U.S. Customs Court 

E. Court of Customs and Patent Appeals 



22. How many otf the cases involving breaking of Federal, civil, and 
criminal law are settled in the District Courts? 

A. All such cases, both civil and criminal 

B. Most of such cases, both civil and criminal 

C. Most of the criminal cases only 

D. Most of the civil cases only 

E. Only a minority of both civil and criminal 
cases 



23 The Federal Courts of Appeals consider cases that have already 
been tried in: 

A. municipal courts 

B. state courts 

C. municipal and state courts 

D. district courts 

E. circuit courts 



24. The chief advantage of giving Judges a lifetime appointment is 
that of; 

A. making sure that they are men of extensive 
legal experience 

B. making Judges of very high ability available 
for many years of s* rvlce 

C. saving on the cost of pensions 

D. creating Judicial teams that are accustomed 
to working together 

E. enabling them to make their decisions without 
political pressure 



23 . Suppose the other members of the class want to postpone a party 
because there is not money enough in the treasury, but you want 
to postpone it because you need the time for studying. Yours is; 

A. an assenting opinion 

B. a dissenting opinion 

C. a concurring opinion 

D. an abstaining opinion 



26. All of the following are legislative courts except the: 

A. Supreme Court 

B. Tax Court of the United States 

C. United States Customs Court 

D. Court of Customs and Patent Appeals 















27. The United States Customs Court rules on whether: 

A. tax laws are valid 

B. customs officers are qualified 

C. import taxes are correctly estimated 

D. cases have been tried properly in a 

lower court 

E. certain goods may enter the United States 



28 . 



Judges of 

A. 

B. 

C. 

D. 

E. 



the Tax Court of the United States: 
are chosen by election 
also judge patent cases 
belong to the executive rather than the 
judicial branch 

also sit on other courts in which they 
do have tenure 

are appointed by the President and 
therefore can be removed by him at any 
time 



29. The Tax Court of the United States deals with all of the following 
except : 

A. income taxes 

B. import taxes 

C. excess -profits taxes 

D. estate taxes 

E. gift taxes 



30. 



A mandamus 

A. 

B. 

C. 

D. 



and an injunction are both: 
forms of habeas corpus 
orders 
courts 

court officials 



31. The power of judicial review is the power to: 

A. try cases that have already been tried 
in a lower court 

B. reverse the decision of a lower court 

C. decide which cases will be accepted by 
the Supreme Court and which will be 
rejected 

D. decide whether an act or law is 
constitutional 

E. supervise the administration and 
procedure of lower Federal courts 



32. In order for the Supreme Court to decide on the constitutionality 
of a lav: 

A. it must be requested to do so by 
Congress 

B. it must be requested to do so by the 
President 

C. a case involving the lev must be 
brought before it 

D. the lav must have been declared uncon- 
stitutional by a lover court 

E. a lover court must have refused to do so 



33. 



The povers 

A. 

B. 

C. 

D. 



of the Supreme Court have been enlarged chiefly by: 
amendments to the Constitution 
precedent 
acts of Congress 
decisions of the state courts 



3 ^- 



The Supreme Court vill not give an opinion on vhether or not: 

A. a certain act is against the lav 

B. an act of Congress is unconstitutional 

C. a case is a Federal or a state case 

D. a case has been tried in the vrong vay 

E. a proposed lav vould be constitutional 



.35. A United States marshal does all of the folloving except seeing 
to it that: 

A. accused persons are present to face trial 

B. fines imposed by the Federal Courts are paid 

C. the Federal courts have the funds necessary 
to continue running 

D. cases that are not tried correctly are 
appealed 

E. violators of Federal lav are found out 



36 . United States commissioners: 

A. are attached only to district courts 

B. are attached only to appellate courts 

C. are attached only to constitutional 

courts 

D. keep records of all cases 

E. see to it that the Judgments of their 
court are enforced 



TTOE - FALSE 



Please write T or P in the blank 
at the left of each statement. If 
a statement is partially false con^- 
aider it totally false. 



1. A justice of the Sixpreme Court may retire at the age of seventy 
with full pay after at least ten years of service. 



2. The Supreme Court keeps unconstitutional laws from ever going 
into effect. 



3« Cases concerning violations of state laws may be heard in either 
a state or Federal court depending upon whether or not it is a 
court of original jurisdiction. 



U. Customs cases and patent cases, when appealed, cure tried in the 
same court. 



5* Courts of Appeals have original jurisdiction in certain cases. 



6 , The marshals enforcing desegregation in the South are state 
officials. 



7* One may appeal to the Federal Courts of Appeals from some 
administrative agencies of our Federal government. 






APPENDIX I 



QUESTIONNAIRES (for PROGRAM classes) 

(The following questionnaires were completed by PROGRAM classes. Figures 
shown in the parentheses indicate the number of students in the classes 
checking each a3.tematiye.) 



QUESTIONNAIRE (cooqpleted by PROGRAM groups folloviog segiDient on CONGRESS) 



1. Do you think the "programs" are a good way to leamT (87) Yes ( 9 ) No 

(3) Undecided 

2. If you think you learned better from the "programs" than you would have 
from the textbook; check the reasons why; below. If you do NOT think 
you learned as well from the "programs;" place a check by the last item. 

(70) having important ideas repeated in the "programs "helped fix them 
in mind 

(52) having to write in the blanks made me read the material carefully 

(26) having to write in the blanks made me understand the material better 

(52) writing out important words helped to fix them in mind 

(so) having to turn in the "programs" made me keep up with my assignments 
(as opposed to "reading a textbook chapter") 

(ll;) other (if any; specify) ; 

(1^) I do not think I learned better from the programs than I would have 
from the textbook. 

3« Would you learn more if you had (check one below) : 

(2^) more time to work on the programs in class 

(46) more time in class for lecture and discussion 

(•^O) o.k. as is 

4. Were you able to find the right answers to the blanks (check one below): 
(54) yeS; all of them 

(45) yeS; almost all of them 

(»-) nO; I often could not find the words to write in the blanks 

5* Did it bother you that you were not told what words should have been 
written in the blanks? 

£gll_ JM. 

6. Would you prefer (check one helow): 

(15) more discussion of the topic before you get the program 

(57) more discussion of the topic after you get the program 

(27) no change 

IMPORTANT: IP YOU HAVE ANY OTHER EXAMS TODAY; LIST THE SUBJECT; PERIOD; AND 

TEACHER'S NAME ON THE BACK OF TECS SHEET. 



QUESTIONNAIRE (completed by PROGRAM groups following segment on THE PRESIDENT AND 
\ r j ^ NATIONAL AIMENISTRATION) 

1 . Do you think the "programs" are a good way to learn? 

£ 881 . Yes Jl 2 l No 

2 . If you think you learned better from the "programs" than you would have 
from the textbook, check the reasons why, below. If you do NOT think 
you learned as well from the "programs," place a check by the LAST item. 

( 66 ) having important ideas repeated in the "programs" helped fix them 
in mind 

( 46 ) having to write in the blanks made me read the material carefully 

(28) having to write in the blanks made me understand the material better 

(49) writing out important words helped to fix them in mind 

(57) having to turn in the "programs" made me keep up with my assignments 
(as opposed to "reading a textbook chapter") 

(7) other (if any, specify): 

(15) I do not think I learned better from the programs than I would have 
from the textbook. 

3. Would you learn more if you had (check only ONE below): 

more time to work on programs in class 
( 44 ) more time in class for lecture and discussion 
(31) o. k. as is 

4 . Would you prefer (check only ONE below): 

(12) more discussion of the topic before you get the program 
(36) more discussion of the topic after you get the program 
(33) no change 

5. Did you spend more time studying for the last exeun than you did for this 
exam? (Check only ONE below.) 

(27) I spent more time studying for the last exam. 

(30) I spent more time studying for this exam. 

(^ 4 ) I spent the same amount of time studying for each exam. 



IMPORTANT: IP YOU HAVE ANY OTHER EXAMS TODAY, LIST THE SUBJECT, PERIOD, AND 

TEACHER’S NAME BELOW. 



QUESTIONNAIRE (coDpleted by PROGRAM groups following segment on 
OUR FEDERAL COURT SYSTEM) 

1. Do you think that having to read the textbook chapter along with completing 
the program helped you learn better this time than you did before^ when you 
only had to complete the programs? 

(^O) Yes (68) No (3) Undecided 

2. Did you spend more time studying the textbook than you did studying 
the program for this exam? 

(76) I spent more time studying the program. 

(ll) I spent more time studying the textbook. 

(ll|.) I spent about the same amount of time studying each. 

3. Did you find it easier to study from the textbook or from the program? 

( 9 ) I found it easier to study from the textbook. 

(84) I found it easier to study from the program. 

(8) I found it Just as easy to study from the textbook as from 
the program. 

Please give reason(s) for your answer to #3 above: 



4. Even though you were not required to read textbook chapters for the 
units on CONGRESS and the PRESIDENT AND THE NATIONAL ADMINISTRATION, 
did you read the textbook anyway for these first two units? 

( 93) No, I did not read the textbook at all for the first two units. 

Yes, for the unit on CONGRESS I read: 

( -~) all of the textbook chapter on CONGRESS 

( 1 ) more than half of the textbook chapter on CONGRESS 

( 7 ) less than half of the textbook chapter on CONGRESS 

If you would like to comment on the way you used the textbook for the 
unit on CONGRESS, please do so here: 



Yes, for the unit on the PRESIDENT AND THE NATIONAL ADMINISTRATION 
I read: 

(»-) all of the chapters on the PRESIDENT AND THE NATIONAL ADMINISTRATION 
( 1 ) more than half of these chapters 
( 7 ) less than half of these chapters 

If you would like to comment on the way you used the textbook for the 
unit on the PRESIDENT AND THE NATIONAL ADMINISTRATION, do so here: 



IMPORTANT: PLEASE ANSWER THE QUESTIONS ON THE OTHER SIDE OF THIS SHEET. 



Concluded: Q0ESTI05NAIRE completed ty PROGRAM groups following segment on 

. .• OUR FEDERAL COURT SYSTEM) 



3* Did you come to like the programs better or less since you began using them? 
liked them better 
( 7 ) liked them less 
(27) no change 

6. If you have any other exams today^ list the sub ject> period, and teacher 's 
name below: 



7* Please suggest any changes in the ways you could learn with programs 
(having to do with either the way the programs are written or the way 
they were used). 



8. Would you like to continue using the programs for the rest of this course, 
or would you rather go back to using the textbook? 



(Rd) contlmie using programs 
( 11 ) go back to the textbook 



QUESTIONNAIRE (ccjopleted by PROGRAM groups as a follow-up to the questionnaire 

eidmlxilstered after the itegpient on OUR FEDERAL COURT SYSTEM) 

We would llhe to estinate the amount of time students spent on the textbooh 
chapter on the FEDERAL COURT SYSTEM . Please check ONE below. 

I did not read the chapter at all. 

^Q^ I read less than half of the chapter. 

I read more than half of the chapter. 

I read all of the chapter. 

I read all of the chapter and reread parts of it. 
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APPENDIX J 






QTISSnONNAIRES (for NON-PRCXSRAM classes) 

(The following questionnaires were completed by NON-PRCX3BAM classes, figiir^s 
shown in the parentheses indicate the number of students in the classes v 
checking each alternative.) 



1 












■/'T'C-IS 















... . 






I:'?: 



QUESTIONMAIRE (coiopleted by 190 V<*FROGRAM groups follovlcg segment on 
• TBE PRESIDENT AND THE NATIONAL ADMINISTRATION) 

1 . Would you learn more if you had (check only ONE below) : 

(15) some time to read the textbook in class 

( 24 ) more time in class for lecture and discussion 
(22) o. k, as is 

2 . Would you prefer (check only ONE below) : 

( 10 ) more discussion of the topic before you read the textbook 
(3Q) more discussion of the topic after you read the textbook 
(13) no change 

3 * Did you spend more time studying for the last exam than you did for 
this exam? (Check only ONE below.) 

(1^) I spent more time studying for the last exam. 

( 27 ) I spent more time studying for this exam. 

(21) I spent the same amount of time studying for each exam. 



IMPORTANT: IP you HAVE ANY OTHER EXAMS TODAY, LIST THE SUBJECT, PERIOD, AND 

TEACHER'S NAME BELOW. 



(completed by 1K>N«PR0GRAH groups following segment on 

OUR FEDERAL COURT SYSTEM) 



1. Have any of the students In the government classes that have been using 
"programed -learning booklets" told you about themt 

^es (27) Ho 

2. Have you seen any of these "programed -learning booklets" that some of 
the other government classes have been using? 

(30) Tes (31) No 

3. If you have seen them, do you think you would like to use them? 

( Pon*t answer if your answer to #2 was No. ) 

(17) Yes (13) Ho 

U. Have you been able to borrow euqr of these "programed -learning booklets" 
from your friends In the other government classes? 

(9) Yes (52) No 

5e If you have been able to borrow some of them, do you think they have 
been helpful to you? ( Pon*t answer if your answer to was No.) 

J6L Yes JiL No 



IMPORTANT; IP YOU HAVE ANY OTHER EXAMS TODAY, LIST THE SUBJECT, PERIOD, AND 
TEACHER’S NAME BELOV. 



o 



(covqpleted by NON-FROGRAH groups as a •follow -bp' to the questionnalr 
^ administered after the segment on OUR FEDERAL COURT SfSTEM) 

We would like to estimate the amount of time students spent on the textbook 
chapter on the FEDERAL COURT SYSTEM. Please check ORE below. 

(2^) I did not read the chapter at all. 
flS^ I read less than half of the chapter. 

(7^ I read more than half of the chapter. 

I read all of the chapter. 

I read all of the chapter and reread parts of it. 
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SAMPLE OF MULTIPLE-CHOICE PROGRAMS 






(The following are saople pages from the multiple -choice programs used In 
the Spring 1962 tryout.) 
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Members of the House legislative committees are chosen by a committee of 
each party. If the Democrats hold three -fourths of the seats In the House, 
and there cure twenty members of a certain committee, in addition to the 
ohalrmEua, then the chairman plus three -fourths of the committee members, 
or fifteen, will be Democrats. 

The Legislative Reorganization Act of 19^6 also limits congressmen to 
one committee each, with few exceptions. The chairmanship of a committee 
goes to the majority party member with the longest continuous service on 
the committee. This Is the "seniority rule" and applies to the ranking 
of all committee members. This ueans that a new Congressman, no matter how 
able he may be, has no chance to become a chairman of a committee or even 
to get a very hlc^ place on the committee. On the other hand, an Incapable 
committeeman may become chairman of an Important committee simply because 
he has been a member longer than anyone else In the majority party. Indeed, 
he may be completely opposed to the principles of his party and yet have 
a powerful Influence over legislation because of his position. A Congress- 
man could, for example, be chairman of the Foreign Affairs Committee and be 
opposed to the whole principle of international co-operation. Through his 
position he could block steps toward greater international understanding. 
Naturally, this situation is not the rule. 






a 



The chairman of a committee is always 
the majority party member: 

A. with the most total years of 
service in the House 

B« with the most total years of 
service on the committee 

C. who has been serving longest 
in the House without 
interruption 

D. who has been serving longest 
on the comnlttee without 
Interruption 

£. with most e^erlence in the 
field the committee is dealing 
with 



In Southern states where only 
Democratic candidates are elected, 
the same Congressman may be re- 
elected for many years. This 
gives him an excellent chance to 

A* avoid membership on a 
legislative committee 

B. becoDie chairman of a 
legislative committee 

C. serve on several legislative 
committees 

D. form a legislative committee 
for any bill in which he is 
Interested 






The chairman of a committee la always the majority party member who 
has been on the committee longest without any Interruption of service. 

This gives the Southern Congressmen^ who are likely to have been re-elected 
many times because only Democrats have much chance of being elected in the 
South, an excellent chance to be chairmen of committees. 



(TEXT COmmJES HERE) 



